From edca08af267917eacbc9fbb8f48b4595eb8a5b57 Mon Sep 17 00:00:00 2001 From: John Stachurski Date: Sun, 6 Sep 2020 07:45:14 +1000 Subject: [PATCH 01/21] Update documentation --- .buildinfo | 4 + .nojekyll | 0 _images/kolmogorov_bwd_10_0.png | Bin 0 -> 3884 bytes _images/kolmogorov_bwd_6_0.png | Bin 0 -> 3894 bytes _images/kolmogorov_fwd_4_0.png | Bin 0 -> 18026 bytes _images/kolmogorov_fwd_7_0.png | Bin 0 -> 23414 bytes _images/markov_prop_5_0.png | Bin 0 -> 5296 bytes _images/markov_prop_7_0.png | Bin 0 -> 62852 bytes _images/markov_prop_7_1.png | Bin 0 -> 62852 bytes _images/memoryless_3_0.png | Bin 0 -> 14862 bytes _images/memoryless_5_0.png | Bin 0 -> 12954 bytes _images/memoryless_7_0.png | Bin 0 -> 12141 bytes _images/poisson_11_0.png | Bin 0 -> 53191 bytes _images/poisson_3_0.png | Bin 0 -> 3679 bytes _images/poisson_5_0.png | Bin 0 -> 3740 bytes _images/poisson_7_0.png | Bin 0 -> 6775 bytes _images/poisson_9_0.png | Bin 0 -> 16665 bytes _sources/ergodicity.ipynb | 248 + _sources/ergodicity.md | 213 + _sources/generators.ipynb | 559 ++ _sources/generators.md | 530 ++ _sources/intro.md | 7 + _sources/kolmogorov_bwd.ipynb | 773 ++ _sources/kolmogorov_bwd.md | 650 ++ _sources/kolmogorov_fwd.ipynb | 743 ++ _sources/kolmogorov_fwd.md | 632 ++ _sources/markov_prop.ipynb | 1207 +++ _sources/markov_prop.md | 1086 +++ _sources/memoryless.ipynb | 613 ++ _sources/memoryless.md | 502 ++ _sources/poisson.ipynb | 697 ++ _sources/poisson.md | 526 ++ _sources/prob_view.ipynb | 270 + _sources/prob_view.md | 231 + _sources/uc_mc_semigroups.ipynb | 519 ++ _sources/uc_mc_semigroups.md | 493 ++ _sources/zreferences.md | 7 + _static/basic.css | 768 ++ _static/clipboard.min.js | 7 + _static/copy-button.svg | 1 + _static/copybutton.css | 67 + _static/copybutton.js | 153 + _static/copybutton_funcs.js | 47 + _static/css/index.css | 6 + _static/doctools.js | 315 + _static/documentation_options.js | 11 + _static/file.png | Bin 0 -> 286 bytes _static/images/logo_binder.svg | 19 + _static/images/logo_colab.png | Bin 0 -> 7601 bytes _static/images/logo_jupyterhub.svg | 1 + _static/jquery-3.4.1.js | 10598 +++++++++++++++++++++++++++ _static/jquery.js | 2 + _static/js/index.js | 32 + _static/jupyter-sphinx.css | 116 + _static/language_data.js | 297 + _static/minus.png | Bin 0 -> 90 bytes _static/mystnb.css | 185 + _static/mystnb.js | 44 + _static/panels-bootstrap.min.css | 21 + _static/plus.png | Bin 0 -> 90 bytes _static/proof.css | 220 + _static/pygments.css | 69 + _static/searchtools.js | 512 ++ _static/sphinx-book-theme.css | 517 ++ _static/sphinx-book-theme.js | 105 + _static/sphinx-dropdown.css | 94 + _static/sphinx-thebe.css | 120 + _static/sphinx-thebe.js | 96 + _static/togglebutton.css | 90 + _static/togglebutton.js | 76 + _static/underscore-1.3.1.js | 999 +++ _static/underscore.js | 31 + ergodicity.html | 454 ++ generators.html | 795 ++ genindex.html | 230 + index.html | 2 + intro.html | 266 + kolmogorov_bwd.html | 954 +++ kolmogorov_fwd.html | 905 +++ markov_prop.html | 1349 ++++ memoryless.html | 771 ++ objects.inv | Bin 0 -> 897 bytes poisson.html | 799 ++ prob_view.html | 512 ++ proof-proof.html | 436 ++ search.html | 250 + searchindex.js | 1 + uc_mc_semigroups.html | 735 ++ zreferences.html | 302 + 89 files changed, 34890 insertions(+) create mode 100644 .buildinfo create mode 100644 .nojekyll create mode 100644 _images/kolmogorov_bwd_10_0.png create mode 100644 _images/kolmogorov_bwd_6_0.png create mode 100644 _images/kolmogorov_fwd_4_0.png create mode 100644 _images/kolmogorov_fwd_7_0.png create mode 100644 _images/markov_prop_5_0.png create mode 100644 _images/markov_prop_7_0.png create mode 100644 _images/markov_prop_7_1.png create mode 100644 _images/memoryless_3_0.png create mode 100644 _images/memoryless_5_0.png create mode 100644 _images/memoryless_7_0.png create mode 100644 _images/poisson_11_0.png create mode 100644 _images/poisson_3_0.png create mode 100644 _images/poisson_5_0.png create mode 100644 _images/poisson_7_0.png create mode 100644 _images/poisson_9_0.png create mode 100644 _sources/ergodicity.ipynb create mode 100644 _sources/ergodicity.md create mode 100644 _sources/generators.ipynb create mode 100644 _sources/generators.md create mode 100644 _sources/intro.md create mode 100644 _sources/kolmogorov_bwd.ipynb create mode 100644 _sources/kolmogorov_bwd.md create mode 100644 _sources/kolmogorov_fwd.ipynb create mode 100644 _sources/kolmogorov_fwd.md create mode 100644 _sources/markov_prop.ipynb create mode 100644 _sources/markov_prop.md create mode 100644 _sources/memoryless.ipynb create mode 100644 _sources/memoryless.md create mode 100644 _sources/poisson.ipynb create mode 100644 _sources/poisson.md create mode 100644 _sources/prob_view.ipynb create mode 100644 _sources/prob_view.md create mode 100644 _sources/uc_mc_semigroups.ipynb create mode 100644 _sources/uc_mc_semigroups.md create mode 100644 _sources/zreferences.md create mode 100644 _static/basic.css create mode 100644 _static/clipboard.min.js create mode 100644 _static/copy-button.svg create mode 100644 _static/copybutton.css create mode 100644 _static/copybutton.js create mode 100644 _static/copybutton_funcs.js create mode 100644 _static/css/index.css create mode 100644 _static/doctools.js create mode 100644 _static/documentation_options.js create mode 100644 _static/file.png create mode 100644 _static/images/logo_binder.svg create mode 100644 _static/images/logo_colab.png create mode 100644 _static/images/logo_jupyterhub.svg create mode 100644 _static/jquery-3.4.1.js create mode 100644 _static/jquery.js create mode 100644 _static/js/index.js create mode 100644 _static/jupyter-sphinx.css create mode 100644 _static/language_data.js create mode 100644 _static/minus.png create mode 100644 _static/mystnb.css create mode 100644 _static/mystnb.js create mode 100644 _static/panels-bootstrap.min.css create mode 100644 _static/plus.png create mode 100644 _static/proof.css create mode 100644 _static/pygments.css create mode 100644 _static/searchtools.js create mode 100644 _static/sphinx-book-theme.css create mode 100644 _static/sphinx-book-theme.js create mode 100644 _static/sphinx-dropdown.css create mode 100644 _static/sphinx-thebe.css create mode 100644 _static/sphinx-thebe.js create mode 100644 _static/togglebutton.css create mode 100644 _static/togglebutton.js create mode 100644 _static/underscore-1.3.1.js create mode 100644 _static/underscore.js create mode 100644 ergodicity.html create mode 100644 generators.html create mode 100644 genindex.html create mode 100644 index.html create mode 100644 intro.html create mode 100644 kolmogorov_bwd.html create mode 100644 kolmogorov_fwd.html create mode 100644 markov_prop.html create mode 100644 memoryless.html create mode 100644 objects.inv create mode 100644 poisson.html create mode 100644 prob_view.html create mode 100644 proof-proof.html create mode 100644 search.html create mode 100644 searchindex.js create mode 100644 uc_mc_semigroups.html create mode 100644 zreferences.html diff --git a/.buildinfo b/.buildinfo new file mode 100644 index 0000000..ace4de8 --- /dev/null +++ b/.buildinfo @@ -0,0 +1,4 @@ +# Sphinx build info version 1 +# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. +config: bfc5d2951501533cf7a34f2caaa265ce +tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 0000000..e69de29 diff --git a/_images/kolmogorov_bwd_10_0.png b/_images/kolmogorov_bwd_10_0.png new file mode 100644 index 0000000000000000000000000000000000000000..a9286f570b7b8393283956a33acea18d84dfc813 GIT binary patch literal 3884 zcmdT{X;2f{77k&FD2wBQAlsBSAPxkTRX{T6sI(v|(73RwNMsL?0AWi40~wwO+K~hh z;sT0x6jYSW2yu;(br=y8NP^WnT%-Z=B))%$+2Q@#@7W6iO9z`pyl7k|QoGrA0~$5mH8Y-oi^EdMoI$XyHj*6!62sw=%)$U^EJ~ zWZCB>*V>&tyAZS?Za+YDj|e2j9*#PO3Oh_BghmiUgN}S1b1W)4C?eb(y9v9=c_Ee2PWQ?cOgPKsFgY(XA%awfK{0#%}^G97m<9HL_$j7SY zos%5)bW7e_FE~(O{$<{OQ0ljojr&>{1$mOo0EXdR^(+RLBWVKmoE8267lYTdJ^*K_ zC#dCA1}z#c;68{clPk@iI0YBq*&UUhyqa(W2`@$LsP$Bz`q0U(Vjo_Zd%+&q;sTgb zyRzz7C^_=LWEfb1-Xj-NR#61>8*Dx$;a%89&mww)t?Lby(}9b5%hwTdLLvZK|*s8Bg}u7m*1jeA@^16&V+&r30` zF1O^TbO=^5POr8{Th?7YBl2>>rH!oO@jHPnoFM|dL^EmfnV}d)YBmZ!K77ydwPpd= zv%#j8t4I>B)?k>aN`|(~bnsz7*H~n{9^Cz>d;jgSena@boMN~*9yVumossy%OIYOO z{qQiunL6sdN<$+n)D3J! z43($X5N12Zti&&JtV$7e6pk5Czp=>>v_&%%l5m70ur8P--=s8=6>&=92yU`>SZSj+ z)q0!0a71XMH0mny5X7QN(N=OSkG7LwlRjIdI=e~M0EiUkF>%u!t3U&A*uU|-yhodZ zzX&GX>d^_JVl^w90@D)LWu!-ObGgzS$ zL1H{-XoWb3#|~0o4<70U@B<}s7|u{1QX^lb(qUQa2>G}}pEjB#Yj#+5Z!`cy@_+#^ zn%5KNDmv*KvYsrKzv&Dx_%w{|f7YbDl-a%EA17_&>+ZU!5Q$BZVz#0IUr*=Sm(N zW8;hkbU1OXp#C!MG7w~Xw*A%D0z>f{u`+MX4V*xTOh4)f^i%RxuqPY3?}A@g`~^8H zp|)i4S@Ya|um$aRiH&^(7kuWvJY7xKHs8ky_S^urlsDAa=C9%)TcA7TNDS7*M5hr8 z{0z?t-n*mqom4Npc@{8ueD|^sbG!^N2H31-qupWGc@qG42LCnN3MF)!Ja3K~VL(E= z5|z|tOU0Libf2!SKvkCg6w;W=uY5j4Q^sMW!j|tCAQc+}7~)97=i>VNmHmj@U3l6@3*(t|90A zKb%~NZ;nNdP8fua|dL41_gYNVpP& zAkA_Vg+z@h`=ZG!vvY|h`5F2(!}qKo^7RY&OAAvSetf&g)+x$wp#gJ7hQ4{v#vg)b z(F8BZ$4+`MFuVQhzhqwVrq7)Ve(WQbt!;y`v`4$4Z^qHPh5_53g{{Uqh%t+ThSE$al5RgDUhC~i5b&Z;nz5BlBz37bO4k*OT z_7Er|X+6~ms*`=9TcvgTYWN!lhFD6xnf}ZyMtvsPSmr6#dN|FO z|KtW$?wM@4^i^spcD@?X;0$^3NW2NeR)ve1dfYR{XJ~dG?IrH$`Bc)wRBlG=ZKx=_ z8y*#7dLt>VJpquh9`@Fu^tMcN*&#zsJ?{*7=5&6uqA-p~H@e;!k~s8Q`)IaQVn?YI zsd1@4sskNaVVT{p!80{{HX# z{+E8z{kWQnmI?}mQUjfjdZJKhs%$AJDaazR>7;A2n_SFCptq9jN>mE_lk8cU;_MfL zLaBc6{X%yQFji$jE9x;{suwweN(+e&N1Y3yQixdPVCtbTB^ zZEKM$ljJyj>>@n7++z3XWF1ccEX_>YuQ4L56FO=M9ZBw$b8WodM+7g(8SZXMXiGqW zUDTaaFzWFQSI;d9fpFfTG5E!k=D0`F+iv+${nh8@;^V+mxuvxudHuyvb1ofi2ghX5 z(2$UjAY)^2(i2X0coh>AeYMD7QypLt@=X7(a;g#N_764x=$Kn%FtX_0IrQec*0RFh zM*P}?i)}<9Z0=fc^Y429koPM}{UYVW)_H(nQ2Tq{H^2_vkorQGXaao4gM1Y10&r^a zuo~wW$B&~fVgoknl>=v)`|sEl0g2xRg!v{HD%z@|HZZOr{Ksg@g>(iQp*J!wf%-Fupgy-M`fS>QFQ+ZS; zuuINhMZYtHlOEdY&-BkdSGlFeT$y7a0zq|%k7tp+p@bAWlnrerU_JK&Fm83?gLy0m z1go)y*mea*Y+@?X$9OpekQGXObz{ITVu{u~Vx297dOc0VfzS4oA)`7QzHrf(!#2nMEd;;#Z~q*n|3%7^qmCZY5pX`Nb0T_Br9*ej(qC5vEE4n&5( zkKnQbv`~d0A#d#eum?Qll_i&?N@?wxeO*4Y;XL~SD8#I9rwd$cT%rXOFd`%}a@==B z)~p$D>`<9p7i!X>H3+)C$yT1ucdIZqNatc)4R(mDZ0yqvp^zd+?xe9h(J ziBt|tzn99c1mVobfTIJ7ogJ0{Z%L_SCJku=aTItH%R}BB)o4B)q7PntIN?$2&A$ok zK>1*_QyxLFN8q`Oqcm*OCfofyOXdc;z+}OLzSXYB*6VJNjA3`^Ne*5Ibb!i~mhc8rT8eD!-0pxOUZ42_ zF5=4=bC7c|v^9fU$Bna>()Qb36*mB?;P#xH9D_nn_yGc5DA?A`|4iG!q2+gDWxnGp zNS_yEQyeuq(w1-dL&oJT#7ZT$RdvI*n%EU5&>d~0(2gL+ZUkGftH|K+k@vwwQ4Ui` z(AwmDOEypv!8iMnlOmU3Z23ckv4^j4HrRf(P<1VN;yI*MQl=BnSn#v-u!b=ENXdVQ$#l47W}ii?y3St_5GYH z${#$S9;msxe_nVnzl1iRQLwo<{mJ8{FqFFT0BGDaUaq5>~`Yy7WV z?N{pM;My}o^WucPdNYmR1_nteWxZ<)BOxVC5#4=ugB?=3CN?H{3?-%@5$^WtO=; zXL{1si6BL3$dmHAnfFd_p}f<2)NM;ooWw7MPB9Y8&$lddkiLelE!u|5;(O(!6(J^z zcm$e8FP@O*d3_#s6FFbHFj=~dMP8d^TpD0$%;>gQ@$_?#iJsgaF5#54lwSrS(`0)E zEh+-~v)CxM+LV8Fo4v@7>ru~EU-f~K7(aY%FyGY(iv1l=p7S|JA(#veToJDht_*M( zB9P-KYGRJIlf9fC;62N^becx z-eo14^3=c}Q^~c6J-jqPPfs9u<^sPnjPrp#)RHqyu}>s~o@qC%c-QeKHLE_7<(*mlCZnf>`pv)q#U-_#(8JEl}C=Vd_u4YS^Zs@+w{ z+e9ddNMD7VOnOX(cW>|K;Y2*npXUm~`(PP6G({a~WFixJ+p*$qu|=CH0V_7wSWd3m z?SmE^MXjwk2}WRQ%TSeOTkMK@c6iJ`#lJacN96YUPrzz43uSLD1aHqQaL8C;-qLO} z>F!`<`g006{u)@iwsi8jH&lpSS6V(GjdPNGyz=h*77o;x3FI`EgB5F!wPMn({>{`Y z;zmwAGZ)AA8L{hl0IbN1s*$<~LFLnLokZTE%g}<7X!7ZJ;Y6irfU7{BsL0q_;{oII zCEiJ~>Tm5{pO*t0Gd_N5d>3gQwd=^X(^~!tC=8+Ural1$9}2JDj@s;d52;^=7L1&j z60tJpPI1&-q)|CQeRZBq$ote-5PPwAaOm9J&Ra3h3~R%`smlQ_l{PUdkRVG*-0c!1 z$Pe-_j?(_pp(*)mRCJDGc5%T2wf+@CDxA6Lx>q4Ea5V-xh~qy@Im?tejuPEin!z1r8Jg9# zG>>0;yy3w;xTQeNmO{kS2E3wmat1)y+$?b>fWKg^zT}?{;~#}C`YHlir7swOyalu-mUDDJ_>F1nY)hvYVm~V z$noV@8JdCdwZy5XfqB-crQK;MLEntgT7yjP`C|&OGOdWcgh`&=S0Aq-eNo@|`|rk< z!t80wt1+5ug6G^`#aBCwAD(G$=)Ywi3Z6H8n>GOkV9TzbRimfL mW;43lPjlWcJSBZYzWrI@AG~j=NM$oF3Owe1wDhCk3x5IebV!K+ literal 0 HcmV?d00001 diff --git a/_images/kolmogorov_fwd_4_0.png b/_images/kolmogorov_fwd_4_0.png new file mode 100644 index 0000000000000000000000000000000000000000..6f5358bba556a8915d56da64062976b2f12c32d8 GIT binary patch literal 18026 zcmZ_01yq$$*Dktg>F(~3R=OLcySpT$q`SL8K&7O+k%mo!AV_zMfP^3*4QKKF-?``9 zJI37u#vc3K@4I@<^~`5JbH}Ku$f2WS>;{O&S(?wZav?q23@){uAR?kYr1#>YS)R1gJe2`%sJ<25gDJda1D z)2s0#n;AGn8W{|Hc>LEa(M(KK8u0KqDJ4`g5;AIdyPL_l#pO{5jy+O?nkvGo7e7@h z@Fb+gi^^Y<%2b3WfAz2l3<^j8{GDNmu0!bf!IvUeDDCKkacRxxPYY^AMQC(%Gz){h zF$5bs1)GyICOWzg3ZYbtk3mEq<4Cx>M~Q4*X;oXr#PVvsVmCS;6I+r8);! z)ZD4x7#^@v5v=T0WZJ(^9422TI0Be9_SJ_N$V z$~v$;7+JFFkFGc6To*aS%GS2fd81oiRh8qs>-yTe$$k#y zrEX2tI)_}*doWaROI?0^pZJ_HAd5XgVsr|UjW*xkbmx8isDHq3;o(e}__C%IR>g)mQ0Q-4h+}^)WNlk@?FflQS8yFar|0yjlrWOANdCIqm~rg|4oyl5Ak6P;ha^ zpcQ!@FVpvCWM{{Bc6MrPOpcAcM4tVtb>+D=GC3(%ZP0A*FDC6f0HxzjuB`1Qxx4y< z1bOW+k0zRjiG)GMbn`foBlI$-Rda16!g4ZK1Ps^zX^HQ}eh5d;Jq{!}Jw5#B$WbXr z5I-U!VzJegR`l-Z6&f*j(a*gobW$Cg8#xbWBnT*zJrlKoRVu@})rkGtUlbC&ay<0) z-O7(*#mteXZD~QUkhqh-NJUv3Oc&ED3Yr8xc<|?_SXo(_{m7T_-bvtl(~O5nDH;zt z^`Q;SnA7dWfxNst^k}KE-epUYj*iX@T!94BD`w<_NqY;PDhhf>V~nNM-1jiob|aWF zp_D633|`(BQ*tlw1(Jrt#>bV8mYY#MJ4BA|YEO=xTDK58AO5YpU23piY@wbmPySr{ z>UovF*|DPs>vwMO|FhUp)wJ+a_xWp?}P5?`CcB}9_$ot?Ec00tGCNvq!jW| zLe-{!H%oR_E(?tx}AqBb*dL5qoWe$>e$mbIH#qE z=q?KdGG!j4^z>JUJs8miBkYDWhzu$9@JUj>UF)f}aRqwjx)qFKl$2KoVF5^(1)e#A zWofx4{vUjJDkrdP*0Ed=t7?L`g%=>`bPC70!a7`}n6v z3Jcz7r2-!UEq!q(@xu((J(Wq12T=Y{usFGWW&b)?C!8?mUu0`l^zwWt3=su!RnOV)}yLrHPcTt6`> zGshzC@BOny{0P8mbm>^5H?B^3QIsBuuD+x4H?Yr*hZVV7GsG~=aL}_XB{DL9gKdepK zI_c2eZeH*$^f?!VxUdgMJ1)Dw&@8PPXxYG!%n?M!_EVZ@pvT>6z$uU~`|UfTx6<+l z&8g?1$NS&X%8#$_3R7$E;BXb?eLA*lvDLN-!E5POtcmDL zglb7YyM6;kP-dPO8Jr=TEar%~bJr!UiH|iQ0qcnD@iIOKHZun&jM>gmnD?)b9CaRt z;zgx$PUQ+7^YhoE%tQ7NJ>IIcLq@mp!L8Q+o)C9DLutbda%l(qZt=r0q34B_ZJ zrJu6@6sk6n&yPJCAl#aUpg(E7zN4#aNXIFlj{d{Y(6k-0iiXanE%rcXeSn(9*+v6?; zq@dfi%_K)~cc8lE5eXdKY@9*zT|o7p}tAC_0K6=J4%1XKWhv zPnspLfkAyh6&2T4Pftb7BOa!%EU5btMQOPjLB9%x4;Fi5 zl74!m+{}AAN;Gt`)7u@MCh2f2k0i<}66ta_s*xLH>%P3+&7|qP-aei?VW@s%1`EBt z?6H`yYF=#jpt4^`tev;~4FjGbA zVO>Fz1>yAjC&-@@alI$j@p;<^1x?j+Nduv^D%_%9EUme$?1hhy4>*3KW)8x^QE`c! zM_BQ#>*>*?^XXBv^Y5EUk_c-Lk=wU^N9#FZ5vW%~_w|L;E-|qc6{p_9E^S7|6I}Ex zn1)$emR~T`{#i2VY*c7Rar#~ej^}L<5}F#F(^JgZnZrW614=pVXBijkw%A|WX;cn* z>?`|ppxj)w(JNYHvdYMoxynC{QjkT(#~-h#QVaPp{~A9C86Wq>ZkycP>8h~U>O0o! z@}-=w>6*b(RIZ3K{_2j=VO8$iCm_BghMGc{`RVSsMHr2!jx6X#aX31{S{a?fAANg= zJd4Y+a=us^AB-1`qq&BB2(CxGTW|w$w&-!+yOsweuQd-IM6_0xf-B=R0tT1B^3ejB zwW;o|vy4w#gdL7-F~$vAqKQ)WlLr@HbZei!SdSqR@tq7sZv-g=Gd5p+hcGGDgJIZ$ zAz5vZ_(I!;tk?K!NNHJeIVb0Va81-Kp}vmG)mkOh3s2$m@po8*BTvV)*kD#Hhe$>m zNdI&#%r(@eY$jmCE_fPTI$Lm)^*1%R3{k^%e7Y`4BG=YF8^ATCWgL#`dAyu23%8~d z`M3CevHJm!fNiH^Ivph`vd4^{S=d0?4QI?UtCkQb^N;f8Z0F(9zi$>M zVVM1%Wz(ypA|)msRS&3=M`VC>Uafmw{~d$VYy73M=y+6xGQe)wZwa2%@(gy{bi?dc zxU^ksr_JJb#r-vLjDJ}dAL)0}M!@@-ywT>PAFwWUWAoXIeXoxHxRd5CP)i8$caKT(mRI&PTKneu!%$;ONT--qk%4;rPMWkfh6-neg; zdSG*UgoTtNkwrBz$f$EUE)p`a;BY$V=qk?}$iCu)*&L=+-3C)LxL}G29V9H3J$9`M zHvam_(#-x#CtNxVV$lbs+)m`ryg7XiwQDd@{RC;lT5C??c+`t=lT-1Yz#@bqY*@^c znAX{A^JadMe$nB`(+iFN06O=C=k|p9IS;FP?{dM*^6_58`|g;vsCZEB;mjWiv#;5a z7`(wi9KW+m^&N|Bww6&BkY~jfSX}I@{%Agw$r8S~EfdYauOH*JA& zgyBLqT_O5#L8z2{&9m;qNP2l#Dh-P4d58?XIj5G#Nlu3^W0}2&ec?$uJ(Li}w$oR( z;BxORiimCyt}1#E!TavQRrupl-5BrhR#%B4cxR%`jm)`SHwo=I?!5bJ*smqEJ))7r z6f41*rYFQ{b?}gd>0&H8WJwKf_!n64*cSFIT1iMXJPsBbQeD2xS;B3&CtY6jf6qm@ ze3UAYo4G=VcbJ>GU?)%Fg%NFtVENcNSXrgPAhj_Kxh%!^^C2{`C!{TpN z#Tfd=7VRM07?q?O_iBgNa1jP%e|neO$jewNp0Y>OCdp3war5HJ=ElD})i?rnPf6`N z7w6a)CX`MSj@a1PEc% z(L6%5U}=NW=JP6*b~7fsnN-V33Rd zS)+?u=fEiJj*gvE3HcE>G#KRT*8buOaO~X*u2#S;LqM?ZE+xQ&x_)ows%GTk!f3Sd z=r2qPjFF1^Yk9kP$%NA3NDFNU(VugbFZkvy%ze`BM{m$n7PZx1J5v-HP9Z`S zjg#KlsD@8maTJ#N8I@W9897IAs53W%?QkpCuVyeByOL^b)x`+)s3C!DY&@0CS)@pe zOh6#YUQ$~-o}`(8QT6t;4|R=7VL5={gKBoL-**!4pnGgAnZ;MN68@zs(UpcsQzkk% z4HWvlW$Jmr%M~-<9+>9}@@IMJol*T{Mf)%RL*05qDm3^f%r`vcnUZ(Cgr;PJq|O(z zlA*?jZj!NsLkPP9fx?t5d}9sw#MF|8I=tR&IU|$2FFGc-^ObA_s>XF0v}61?W>oaUQqhKTqN5if>$19v0S;bs`CxjW61Hu=kzB2 z^+@dPOv;CDXrN<>dMzhj{1J{i9@R%y4&jKceO%9*D-ZXjWNRzjl;J3btTlR-q-WMReCLPbH-HU=R@P30juAY z&Hl>O8&F483aw#bnS8Gz$Ge4gyb8oh&!tbk8CAnO66WSxZA&52JUi#iM4V;_scE=p zvzj?*S0UpzGYf8+g>vVd-ltL58&AGF%MMG`?+_V2Svz#;H$V(O;t^L=Y^TWaLJ!~> zlz&4PwBcXBO^$tD=U=3B^Xq<8dI;dbeDTER*37Ifhf5>miA`T>?CJN~$QgP+=@GWZ zZfC5*bMVJvu2Am1QQ%F4_ck&+4;Sex&*x6CoUQx$0Z%#*pb%9$hToNjGiVH|C*j(a z7?z`{dClw*1^Es7Ut=w@MIx2Sv!r>N-FB&kA1(;SKQEvj9K;x8TD}<_#Q+0~ZaGxd zKUHXCZ()P~4xVGIVQ5Z!Vm4%yfMIUn-|;gpF}Oc=ZYgx#F#r$QFBr{09_~ydQfPZN zHE`ozhz5@LB$Vd@21D}tzZgTa#9i9PMx=VIKK$;)A#Bx%&3Y5)GkXdG8 z`CAb5gunk|OXK3thQ^^A>s~?JG^e>F5*l#>tc;A2z2vRMZ|LhjMuKTfeH~7V0TL^# zB8%~?ub|QTUA{cS)|qpIc~70{hZ@muxID~Y(Ueh=Dk`!#&1v1-{I3s9ogcxe>sON{M4Dk$H191tA%7Uz8{H1 zp%=C@jM~f|^&(gXHqA0iD<$aR5c2qVMak*qqzYDW6MNKz-CPBAL!+TvIr2dyJ#1Q; z@ZmgpFK80-`@MgXva${HyV<3%+oS!-fvae)Po7ZiA`h8QL?Z3zkJtSP3nnY&Ur+M&%A-;*VSPG2RL=WiI?Tw)yd5M+FekVNC&Awi&L40z+PKUU9CUm_Sr^03k$q@ zFpK=@S}H>ElE+;4%C@H8MUXm*=d}y_N!y;OUOf$}uCcbBE$2K6Uga`+OAT^tgzLaNiy$=4)uoG(X*R_|fd3Sv|%q!!#atad;)J!Gd*d z{%2J`WBcm!>k1{qfLo)GI1ZKJpQB(k>d3xb)uU17KZv?FVy8sJFbV-)M!4U}=xlp7 zcX+5#SpobD>y1wpYcOX`mj+qlXM^tF(hbj+Y%ab(i zV~E7MbLMx$ix`R}ldZ+s{C=ivo=Yk;o)DQB7zmfB{)wF`T-Y2yc2EiBrlsY1t3;V) zq$3#^bTh`s?=}!&n|n~Tl7Kw>(?E^g9tCeV^D*)W4iDkY7-=)J5@SuzBhAH00K!Sn z6KZ#@@&Hq#l`>^kFxTx6bGwP(JHiYOv3P;^?fqMIU|UD{lmCI8{w>Zl5}}ri`ZF(N zlQsBH^`@3#~9Jnf88^y3dQKLEcS4-d~pKPDpJ@95++ zkbHJnA8wfqGI1$nLcF~x2XK+T2;zJ(TE($nM&M~zTxpAuvT-@xBt|3ti>*^TG4w&V z#K9P2F8HYkCrV29b7c6vyWK4`-J1C3<_1kEZn^c0>7>mT-^8yPP@OF&*OX>8z$;S~ zy`i8CXo4f<`zPn$+3km3<}Zt)G~+eeAT;0p??fbwTB;j*abV0!nPHRuVGb6ZbiU8@ z(c*hCuZ>0eVbcLzTGhc~y`+^b#~^LeYWUZ8q;|#_G9gqivIH-A{r`@t>NWq)pWjvE zA>KWG3xy4_OJ@wFVdR($tEws<`DPzt()9_nT1FkrQf|Q4mnS_zJSsVEU~hiRK7gA^ z()>}~yOYA@efp-!QJ=CuN;hajn>&%z=`A?J*@w&2D{n+g%eVu~NUQ@e=dvp*y!XFw zP%F54{~i5&(`h);kGT;o6@|hX^mqjhra7~8Hl+l0YxJ`|JM`xRw>!Wmi))8hK9$~N z**$TM-F&`kLM8`wc#{G?$+1a+~*|efaaCSJTjmzr$jjcIdzb~H0@Sqg*H6L3=V6e8!OIv3Lyj%AtvmEk|Av8S-iDQ2=(@r*>MfKv@R`ttK9uNPXivr7BX^1;ylwJQ^z=eu1 z45@ZtM%7C;j(`}{CaB)w30tCk7Po$Fgs=p^W8A<7FfSnA=E zF4=Vs~(hQdwPrICg&#h2(-iyg~-#SprgUQh`Y0I zFGW(a$~J0b>NlA-U&0q%Pj6tsmuZj{;jX+cAN%$PMgr^fV#fIN@$Rn9X$>2UGn}Pa z>PfYc&x_~9Rav_pa&aMDJyxEzEhW%TIcRE|S;a6tgUj`9`5h6L1-_e`5tvDsc=v#> z(q^fy0at6x&&Dk1ZsiyRSTX?Ad81H+`9fpF49^NPU6Ac{Su7+!;4W^e?Y*Y(H5KSL zGMuJu{6+W=pPt80Q7~{BmfzA7N5-p4Jp{H=$daGjN#Ru~s5;_$N?3x02rFpu-4K(= zW~OPAn)=tf+OCJi{(&j()r*pYBhc3=s9*s>jkc-*ag{vJg7P9s=5h zu+wMXlQF8AUf&`G!y!FFM^fVI&HIhtE^9q?5!jS4 zp*ZrA9(k^PSA6ktI|F&V(RNZ2Pev->Q+-7N4TtaLEoDu2JM~avm-?Ywg{n4{H%k5+ zPiV<;$tl|SwCJ@Nl1|lP7)Q@h=?3uvfmsu@Nhy;xY?{bQzc(7&L9Nz=P59o$5DYG{ zMEGy{IbqiFDv2;|5O|5U&0Q++V9u~Da%iN7#pRq&v(N^#RbP0*$mp5*?^zZ92`2m4 z%ulP4UKpjuWV^X%6&!^3dB7mODYvBNa?@i~04b zHvjubMddQL3}hwRfY|y1nYe#D)h++bPF79i6(A;I)fnpFni5 z?D!G-6WjE*+4C4%Dfb@T#HW~%n>+RS470hv{#@_pg+ai+TIs=D^gcrYn}ViVwZkiK z2)G|yexACo?c74uwHqVAvo=-UG1*b(6 zUCHDp2t2aeee)^32f_eewPGcXlMz;~W+@7n85HUIw0~u)#uyGDa2|3>3Uoc69oXQ{ zg!>T%WkfvoY#JIkT!Ht>z*V81YQz8683bz&qg76z#f4n(o^*TSo-$vsnLVi>-)W6b z_)^Ojjr1!{TbmOoOr+2_lMO$ssB1KhZ)*gbFxRK#m9+ZsWX$idY8PfLE@CVkwfhiF z*dhb0pEJqXE6p{@1x1>ric>XfgG78zIrGKI6P%pPHYNccggQ0Cq5nCze|TNx?+c-n znTSWfJQvY+5qJMHpU)EeVVI(Wc;#zO9}?x!Ogr8*_82^aox|C79xx3Z0Q_+jl?*u; zs}M=&ubu_q`yXZ*N`?Z)$747g`S9$p*2o@*P1a#V@0YQ26XTE)uaj^rGH!R@%i0U+ zhufhn*1|;+h=u>z(V~Wy_@#K_UR#EZ*C0MI5!bda=r4+BUV(=|iB;1fH(XkQ{bVi~ zIOq`ok_r9w{qU`^$~8Q8_s{O&ZTNIp2ui*$)rc*`502?f*Up%F5SmQAAerizC30t2 zt;j`QNDjvTt$xgtl9Ix9$$Gup#an7YuSbAGP7yDTmW;p#l(~3>E#E|qUJp|n^kzbS zx24!gj1~W*+MgoxgVx4q8Uhi&3-;r~9}3{zchB!;MJ7)6!B_bQzlKT`S+arSyvA6p z|6L1D|LN9I_bdrqqm54OBQi%<6U;L~Xm7t1W2WUHi>Bl`sxjar%#Da?5>r`YA8g)V zGuKX5KytX+#=P8Q7cNJk54gP&i>AW{upK$q(=4TdqxB`KJDhHCD~56T+*zfB3pfV; z=iib2ueSn0@6Zd4DFGd8hs6;1wnO-&%k% zn%tnk&-2A<@%lCytPnf`g1)_T2|Tt-0qWnp&Z3Dha5~S168dbs&ovDhRr(R>f)NQ>FSx?u?O+0p?AS6Y-ZC%hG>_eEo0R%5|B3$C9wE6b zc>WAc%~!Tg{fXoP&py_Xr{y~Amu`D}j3{w{S1#CqUMIJXinEjvhia#{KvFoH%Q) z&SH>S_Y^HPU^gzuML-V`UNrRB}v2k-Om%8Q*okkmAv3I!i6?FgpW=(w5Vjp#$p za^LQ|6FdO+;(8vAJ*C2nQ3j3jE=D-py__Q}YbuhEae3bw$JP*^Jr5n5l8iba^@a&- z@b7OODB3+A&~O<&#YL&~g*16i-x-65DsE%p~gw z)nnmIXjRieZKyK4KIJLXttFrcY^?_1>KiRyUjmIhwdYw1R+P~}GNN|MTmqSo2nvc7 zpaMLr6{qNFP2#{=6U>!Yq^F^cpCs0eZ98$z>0!_K*cDwdYJX*k)FHrLh2y`(*y?%Og?~-kI4h_GVPatcjPjPjIHbe# zAw)DlbQn-pa+NyhOv!j;5pFGo5QY>n& z;B}@5Mnq2qYg#k5fC?b}NpvEo1SE(`DJ}@}hWJcu_1}Q##Vg@|dP4381%G7b! zL~F{2W;CyjfP|C&262}i(iP`A3Go7wD7v{2>A6JM*0Y2M!t;OzTFv&SN8XX|mlo@t zbAL5&E|l9Z@Id) z^j71N^mx`3`J~7do9Bsev4_AXuHT=$#}F8xIwb{Vtxu25@`{Rm*JpeHCF%X$49DBv zWf6yH<;%N7wc@8dzP3+~M8x6wgX9cyq z{0xAwFFOUH+Pr+C%>rlDKSPR!wtSr}y{QZ3PxIAI^W{%zU_4j%yTTRaM_e>CL13|v zl)~E#J&}?F@HJW7_GSi7`Z26yAeh>++K>vE{#a@$ovr^$C7w&KrQ_pW77j*{hgiY5 zl))nU;igou@U!gpA#m82BbSvHNmv=_#E!kWIGoev+uF-Z0zBt>Bry%}YSn>wXM5XS zB)-p@KG;=kX?7r2M1V-}0u@Yz_LaGhR(k!^FNp>gDtww$DjIEkYN^3ePe!u3EsZve zn)JiM5REuUFe=*kH~7e+0w(%e5B-eEhZb1G z((d~M)n?AX`|m{DR`l5u4W%bp3F@TmU}Y4Fs%!6_jkMmKXt(~ZKqTTlG5}jys~e5% zGP)H?OFjGTH9iX-jJz(}RVsR1MMbhTJry3ze^g6mEyKMwjhUVQ@MV~9hBs%EL<4HX zcZQm|eEu#NWa4;$ML=q6ECEk^If{~5Q09Hw>yccDj93i1jKq+208^90mU-H1<~mx*gg$&SS<;pT4$H5gK^v*^Tn+Kl=B+YF{c)tcbDT z&sME)AG%el#IfL&K28%q+&$s0bvz-jwHsG?DBumG>QXjR(ullUCOF#2OCo;d6h?P& z#jK-k){89E&4H7`0>g}x%m8a)BAiI3EK7lhCoa*Qzst_zXnM_n zEcG#?h&L|2iVx?1 zdYKib+n!8kdg8DO2K85$ttvH17fge&Pg-|SR=qX|ua6pq+}EVEjAb8kSbgwuWZpk6 zKizHA0wnN<)7gY$wx;@Nev!IGO*n$_x{DUCMbx?l{X5?0-9GNwB64*+^Twr#hej#9 z;uqo8`R8mC+a#feIe3bgBC`a;i2`T8K6e1dA6Bm+0y)b$|;r*|5RM1T8dUlu7X!--ZMxpU`BI z0MGSn`~>VFTKD}QSh^-3YdjRN2MJ0-h%QH2CPU!bh=jfA0k;%Sz{dS-^H+j0Pk%88 z1qxHsPy`P%Z=R~sBvMK$=v-@w@X1drW9h@ zdsAy_21g=mzCEO+-(f+^fvbP|=JgL0N)n{1mjg#D? zllWdX@0s0fenr6Xh+L^dKHZ)j&G~z4Dj4CvR9Qg*jg&L=tq2oo?% zzqP5%Eu*It6__9v`OHT-l+OGTXqKpEt1F+719s1PbhdXc0}OClZ*F~jF*04sDq`y` z`Jg-4oOaV}?T-O?2;O_NLW#L}$+?K~7%&BJr~DyOD+pzKk<8loZ*>-0Ei*nGH_4M!_b z{LiaH^Zp5KO}q?0x}cf+BCZ23?+HD`Nq$0dnMWvMEN`Xh*ITQ*&1=F{G|IqvXF0qc zy%G2ldVjOwVEiUw{{}A5nstf(@cc$iP0ehCeaUNMEsW4)F$4j%Atg5#1Uhy#@Br&d zQ}y&MZnY7X4(xL(t~re12)hxFxYIWHqSqQz|9pu93~!ThYX4*!metIkuS|WcuH?ay z^jILdvM5LDxlauC@#-o&l+|OT#q{3Lx<`6s7wd%O3^**;sl?pJU*j#z;q3Mm9FI!@!5Ds9Cqg+^$AJ zsGgvrL+4jRItlPtP&qx1lz|~fSaq_z9D(e?&6Kg;u=8!>0&T5AJVn?4P3*nB`-iJn zXZFg?4P#Sj0;V;tJ8bMm2qNAP$N#?Jg-KGZx6qG@h{yGd<)bD`n~>geH~ly0mxl{D z#Kh%Uf^%75CwV_y?g*_z0&jL%p93h&nu64?8grlR!Cv0`vne^ht@D`ZxVW%2CppPZ zbdv?_)@;TvWw4zlQ2nCqnq1K$Iy+-mu0xgO82TU=P`vVs&|o07;eB?it4!V{7Bk42 zk9zf_h;H4VMd2E?x9DR@36rKYEs)rW^)EKPO8c^vS8 zC+!Xbf}W0{9VhmHa5NY2=f^}w<}E5EL@Iewb!?2mlHtH1ce*$}7LPCE1vnF-&Sh zXSJ{o4ru&Ep@#^Reyf4Cf zGDFoZ^TW^(=IDR+6i?n(WeP%`eq@Zt`y?$Jm9ee-q&DAwC-&U$Ivi~T$sW2XO*ZJF ziTq&{vYo*2f4KaQ_h9U-g!N|SE4u5FH6az>Ld3m4->MS_goC|}4jCxnW0rRiRhMDD zsVtj|lPK?7Gn4JHRx6~r#}m&GNJZ6z4;S|6@AbydLSER6CZIy{^YbAMtcNE*Qvq>l zx!hE(SGNgd-TPK3cKD+Wv0Z*o&_r^^txFbwINE{onw>0hB(kTcV{WoX*WqX~xoGjl z>WeR}^xwW?#g8KX|iNnm0?;unE4+;=aq@B`;m9ba}@)+8 zMBe517s4c(XOvu)lF6vMZUi`_WMqV&?Kn?2W#}X1z&3 zr|1P}R)jH=`X-cd;LRbQ|2pmi&S>%{`{?fbI+tk-;^CX)tMPSX!%`I-py1_2qwGcj z8shKo@3T~Hm+H2dA~B337EBHk0BbpLN9)udRr(HFoI{WUt=H19bSiNzjmj#hn0p6>Td02%mt>?^0X_TW;Zt>tu|ux}Gqj?fd*Gm;YV zeI0kZBctKNrr;ze_sh1%ni(i^`+zdXtUvPc?KBu&; ztU6X9J0viX1XeY!6QB#*-FG#^b>=U54DAa_$>%Hj7v~nBH(as%XWmxNnqwBthNWdN>ScU1~ytW1fp+V}=8l8ez?XRpd8^7n3`+)v-sLs67<)7nG#;1V-U5;he0PFbq zw5zM0pd>J8x}Db+m;=KO9b6Kdp68cMk1wn-YD zd4tAI0{7$T;mP@3Ek8}uV>vo8+IOQ*~lFL)Op;J zB}rF=-I6suL`_QzK4}=YzL*}Mnc-lVm(Kv%$j4e%`s9^SFJlm1lHA{)m^8=xLHgfv zMHtVZaCsIxLyxULQ}@|<)bS9`&dy|!mDyh+6O1yR{2#tEQ9%#UvpIJoF)__E`a{~$-@GQi?h;9B zxd7ua)`9l^=Ij{%-j6Q}5?Igt`AV5shJ-=pZo|a~ZM;E}AP}6t+ME!|{34e$HdQ)_ zk6m%psg~vmE0wyFA}y=#c_$(=UULa(=kxEz*S|hns{4@@M$wy{LE-KH(fM|9M`Ap{ zDu13IEK{$h)zP$0^Q0RcN~YsxePB~;JMFVl`snsD65TMSQ#gr%hbW}Dwv`7NxFKAg zM~L3Suz;*l+fh~R6*_(a>$ObH-ScBiqlswm2$1JBJ>A$80j~uWm4VE(^^XM< z6s!gpxF&&Zj5xUc8$Laq3PP)PQPHtCVU5=$ScfOe1JXjzN7pcgVNw{|fK02Zi#-AGScTkYYi2eBqv$VE|0UN`^zFAd%*Xdx13uLJ#%%@Q zNca>gIR5dy5D8z6_Q@nLX4o<+N14Tdp7uDsfd}OK*Rs_fb^Y5|;}6jL-ONT1i$DOu zXIv0KH2L`F{qz00OV-R&Bzx&}*spBd`zvct2Ii}WX?-_jhhFltY(#7)iDf_U>1Qn% zpc3CW;R*u;!4KM3aD3l@N9DVdefhf>(8emQSL+{`n96Q6a0{QKR>GY9cIH6m3dqN? zsd7!ti^GM2zW;7Bn%EEfiSIoRJ&Czrky_keU-EQwysEqZ&{go`;sy`2p0Lcchk(L6 zA0Mo&6qN{!3Hc^Lu;(!JdLEewI7G2NYM_r#mwteVo0y!<+DTy5S&tDI0a-TSh+_*j zyRQCiPs#BWD5M$4zloLd??lHKz6lz9VxwlCh-~FI{BczroR=2b%|R{tc>nIma|-X( zNVGghEXP`kAe;5o$qop2A@jZ?36@RJs4z*WpkZg9kY8({ep1W890&F})w14ZB0I)< zEW`X_e+nqzAX?f3Tm3RF9r&$lqWn4B*(#KP88c!hX3xY_u{~Re>-i<6p5lMv0CG)0 zHq%4Pc@W#FsFl75VV#CT8DgY^3qKN)n+p*Vq6Y{7t_K-q4+a;BE4S}0WkMmtK!Tgu|NHmv=B4kX@*;N1=g)CI=fAe%IJ%+lCq<$_9F_hz z&qxDoW~F;F!P8U2mLb`+TX`Pfgg@MOKHY^pk$aQ2bX*6CP!PMi4#F7R*Sf!4r2IMv zLrye-Lx$m3eu|Fa`SgJJbs)~!7S+2EN8J$tu~h9Z$LeOfHNw?e3lVS$;~8_`Y8T7d zn0*3xC2VY$MPKjm$?NwlGtj5Lwx5+h#3__ipUuy3D}%C=P7GG>Lpd|U3}*J6_dGM5 zV(z2_>zWUp1)_d`6Q%rN9mZEjPjTNoUF{h8THS81!{H%>*vad--qMHY^}ZCVz8jK3 zb35me-0Q`yDF|Gn|07R2$Z=|5MGCkrkXP5{$uDL_YK|TIn)+bs)v=qZYrQ8>-R`bhMY~>NZEMLo8>kJBBAWe))T~V4Bq8;EmvWcx9p={;-^~ z?$Hs1R+CNdCcW>z`DNq(^nlH3IN!25>hU-#gAqlmgCGYB*H|KnP#RLAfcr5NG)KmG zrFYB77xaIiv7zNWs!{A3&enT)SkYRNw+^3-AGh;J@Zg8J(YiEu!1q)34px~AZGel3 z9&U|eL7bWWqM9gcerS`ffj~w;tY4`0(rBsuprD|^a^}l$V?eP@O2^vZt4>8{8x50} ztrsN>uCD)*cNTVQb3`{ZYW+1sr+aVi{@egne~|gXba(5oPwm!0x2v87uCA`?-Zll1 zA;flQDm1YfP~!Cu9`BGQH!*VgbpWVPvz$q#OGnvhVB4A&iX0qVs(ZiZ3Np$3uhurX z8=C=%DDdltxFli;+q`yG4AqOeCC6ksWYN5&UIl4v&D8|`PHUTLgZhJsi^)?AlMOQa zNKZ2?KIdGB1Oy1irLp<_uQ^3;_i39P7BKkMy)BjOlMvS6e-$aDxokxc0t-hu*PgBc zg1=!1s08^kaRj@M@t~HKV^9%jFgL3N)>l1+6ph@9WOX&L*?6wx(ugMou8* z8$zP=SKC=p&f+;uUXCU-uHEP9D%wPqCy;O2) zsz!ru3-P`M+Ia+_3H0&djw|TV9|C2}^M_T(tG5_L0h0LVJZJN@$$W{hd?pl-p!e=* zu^vvu@819*42{hX>l#+tfK_?>0|K(F;eIdFnig*;YR{z-lol{y$*})rpgI4$7Zc<~ zsgU|+0)to2*>D6z1_Q%@jRyI+81zp$IXMa40e3VaA|Oxp|3B?{Vc!#MqqlPv_&PO4 z`nwx1TL+Qze`49w&@`T^247s23=R%Hk3>vNjI0n+qh$Lvo9CZ(0MS1$?2G>Uvw^Ab zjK>>OYprc;wxcKm%g&B#TDMUE&}dw41|~F!4JRxuI-XTao`Hek6FVK;fyK0{lM}m! zMnzz(S3XEzEJ8@o_-{(|P>Z61LT|n0u)P+g+T%|u&pDhF?CTDLhtJpm41aKMG1ww0 ziGx-dcX$@}>U{k9)9UB$?uT0g6BCnS39^Y}5C&DR)Tt(!Ba8-1PIe1ZUhFBWh)&Z$^Yr_oUHBMr^?^5(rAmx{pyv2mMPPhA%1Ip z8(lVRyEod}Ah~jSmj%QYc6L%g;xtGKUv%Uhi=fX>gC2oI_3rz>W2sO7Zu~)}SF$xR zNEY68>)kv5;2*SDXKtPG8TeS#LPDfq@yM8)Q}a5lMo)^~p-Dxel}{Z5=?tKJJUYoo zAP|JQ=idd0WH)Y?ek2r&0v#|w#ZXYk!=|K60zCz}(t5mL_-HtzFvw#;0(RelJ1nF` zH7_wd9DY1kgcOyKU0PBS7F;p;dme>JDcj+B{B3peu(k)IbOr?Y(9lR;zi{q6lP%A% z@NjCe8pUxxTEs>tc^!RGYm5jc$zpV4t%B)|g1jJEVk&WsXUE!KYGQf9Ly!dd_vh%y z$Vi!cDdqo@fL-~<4_D{PlW)}p?s@e#IWBP@tw_H)7wB=9 z_Qk(4tc>|wONtMwvHwPy`dHWo%ZImlS%ijIGR4IoQ^pdS83O;6&gZx+Ine9&%jxxk z2!c~EluLyM1&^NruqZ$w27VC$B(B3p^<$^)QvLU*2*~D!%Rn@4S*!h{vJ7Bz1~%ym z#C;7Aas0DnJunBOhp5T41(ayb1bAKP6 zPCEV+$R$yL+;j*ep7<3aFopcC+tQ<31Rp%w`)C?(j|3Xd~V+KN$Rf0p7{rEkFI9qzirf5bS z7IUWTOZ~r#CCVb`+oYRD5@CajY5Jc6 z5wUWl`0ahO5aDJ*iL#~ZZ9mM2hyG7Xruf4POQP&t9E4ex+$ML<#I?1h5;Cz4K0e+4 zrp!#IE1#vPv9S*Af8CnNs?EyD50gGVL0eF{Za> z%0;v=KvSxqKxCZq3At;54gPKes6|`(PpEKd_TUa3{LZO1F}j4WvZ;MT(4rBLj}MRy zP_aY8(#92POGDI@fb6P)DBFrx#~4x48hj80AKZ-)B_)7-#~4k88;2e43ktZ-3&L#+ zht@8sCoaSH!iE+2qEmKtyD)qvhmp(`6fk%?wdP2Z^VKAXlk&$I3;6&XxFu z3>IA4MM$qgN}n|q9zB}UG$A6RHX*n(9o(5ml#Pc0svxp3OYP?3XC}%FNtPwbvO+^s zaoFL;nubASAVv+F5o|>ZcaIEF7#i4o&BzW%O%61j^~|vv_O%9xrDvc;#PD!w0-~p9Ye9A29(;Ed`LAunwh=EKKG3D`DIYAZ1QyLm zP7rc+HffrC+M&Y)xIegS*(^4Y=9g^DlQnMr5iUH$3v$6FRiJCu{Aj)mTxl}xsS((& zUBIo>U0WYKU^xJE_^;y!R4#SdTh!XgJ5TcX53~Vz011cW3>gDO><5bP0 Hl+XkKzFWl{ literal 0 HcmV?d00001 diff --git a/_images/kolmogorov_fwd_7_0.png b/_images/kolmogorov_fwd_7_0.png new file mode 100644 index 0000000000000000000000000000000000000000..4cae9c4843679840989905d2c053c3467ae0d241 GIT binary patch literal 23414 zcmZU*1z45Qvp$Ln2uOn}cZb7=cyBq1225F?bL%O@};{49J|8wu( z=h=_2;ahvn`expF=biZol93icLcm3Uf`UR46BU$$f_gOsUJ7uq;GLo)#6s{Nv>m^g z0v!0~4yPXozK6FKRkeeHdWZhG!3H#gBG zax}2DGqJK{qGzOMq$M)8x3~Vn!0>;5PH$yv#1Kyy8UzJJ1SKZOr{J7=u;h}8IW>3k zxWb{h#C`t@?zInPW|msNM^3)YJ&CsSAxKq1l)zsY8}+*iOZC(ZhI>cfxzw z`sv^rXo7SX*`VLQ*C&iubuHOy@lv=Rf*?aAMirE=Z#{V|BAD#2S>!bxE*nS4W95vK z^G%ZL+{TZgn58C|>EP@;C&3pru@# zf5Gyv_0PA5A8tpz4DT*>q0E<>;wts}=$DKPWNCvjfjuj4VTmEsx8i+D^<)3GSXbOlQcYUw@S6!(vOy!>yNOf zYG`xS6O*xsLa^x8H)FoRX6EPn@1(nha~`+EL`S2;qI?`mGD`1{`Nq?@uw%Tt0o5CZ z*QaWfCbTn>=5gEcho1PhtemUE$Pdj>()TIQ4gthYR)cYHA3)4h}3^TU(ruEiEmn z#l^)-JR%Z=GnjdKdFCs>TmRSZhZAs4lh=VAQq#~l-MJVT7+CNx#l`cKQ-Q1gjb#ax zWQa^uMg7-yMDT5(X_U*MSKHs9bk|uZm8mJ-pM%dgj(FpwcGr-9vmV0m0rH*qe_fIz z9JYQrqy09sp&`*`qxZAhmAR>@sY;zWj@R@3N)o$`?qOR7m&;j1zE{nK4OE>4MnsFTB)$j0u;kulZ$Hc<2ozZl9brRwA z@Kauz!|x-vtNz{0bWJHy?EK-~p6jWD)F5NE;c&LuOu2^JVQurHQ@?&RIP{a#K0*ux zzvPDp;9>3FZ*IE?pW1Z-2&OmtjMd^z?N1&4RJqppmDr@|LD=ZzimYdAa_7QmwszK-rto;r*YMC`R|- zv1r+&_@-Gkb><74a=V`UQozu*yEsO&YCDXr;kuuo<=Mq22KH!poNfB#cne!rLuoR5 zc7J6*x-7oyLai!!dU+h4F-lZMQ*#(Ne_?q!IabJ8(~&e?og~G_nL3w;3Or|nIY&<$ zOA2Z3KUwlc{Ra!nmVeLlVqCsR_t}V&$B?22%}<|xdHLOst%lBvYpg_{SqDHRyYF5MUm}(6-o%WAqTidCV22 zHG*r(YC)=?ff)YteMsfXSQgja4#QlF3*IOXXOoof(iEk7+cRtOv|tm8{DfNPsOO(; zwYJ~>m3r8=!z>gKV;P1{0|WLTXvLL^EW-`q+}MWr;VEUpQMMI@(}1t;)8sb)z$u4Z zh<{C<=52CNXd6;%tKD>xTZ|hRAl>Mt(><6Kz+uX7yOyFt%@;GNNQYC7DlO^=o|Zvp zRsFS!NdE1KM!^3f)9YE<0!z6`G3lu}e<#s!FaS}?jCu)VnHKLa@djvN!bQc!*;Q51 z$;r6U{hJZwM@Abx+)4Hg|FZM&ZAIDgC2z$Jue*ABMDq%@s{Q}nU5a9sTf7-7_(owk z#GWn|6CWx0rP*~a^ly%NZRP~L{Gh}xmOj_VXIUTLrX_A42%k1pkD_wxtHed;jmWh1 z_eFUp;@5Kl>qkp`a0ki8hogNFUsI*i#2X&3;-y$jiZtWQF@k*Q=S)>467*|ZWV|$$ z2CII_m(l(?VlKGfJ$_=~?1b#;E1W{aV*G$~mY{gX=3UMfa7&gzv$j4G~+Y6AaD8zW# zrP^=zPfUg_BOcdjd=*+-%})XT-e&58UmQr}wEQE{XX1 z3B&iiAOiu{KekbTZ&fKh!n^^wsj##(x2lSf1{=|sA%(5)1E(W#g0?3MnPNP1VQ&J{ZK^ zJw{$)=r<9LGY_{AaHz`b*^F?6us>eB!Qs4$B6xX^r~#brH+!FeN$kP*$Z*?4wOGQoV~v_b0w z-yqylKv8W8H=Ra$1YIAuthmi#trdDm$6_M|+x@YguOzCEj1C$JCci3{G*{(9-$AYN z=lT^-T=@d}ttF??0=+(;=?8?vavd)tS5sNnni`YjJCA&p6iXdbi{eVpKjxYZT?|%OZ8TsSsI!i1?(0jlAAd@9w<|M!JaJ;K6yCes4UdAa004- z<1jkO>?|x&t_QU<$lEdB>dq$RY1A&}=Y)ZePLGkPmQCR<@N$$rQMi@4#M3{fFIcLN z9D8|T0Suj7-+_PsKARS=r0F&EAM4zH&Lu2NXL+f7se3$%)UmYR9-KAo;pSBZ(H2eX zStgg@2D7N^Jw}Qb5h68>sU7`9VP&$IFIjXuro39Hh&w0F^VUgD+%ka6v!;93oZr-s za=g;^ERm@Nu(frfLdzoG`0~+jIbSVE1q7-TJueil=(N z%TDVby86`4AFr0w9^Hj)&;}et&F3YnvR_kR32?X_b(LCm`hKFK41_oxHJ61`r0hnQ z)p0iY6|qpaR*OWk%8bCge{a(puK8!PUxdltI3tv+Kn4K^K?e)-$Hr>}&iNXiA!aHV zr)3VG(~S%e)UKbMPk3;G=+wbN(;qy#S$@o?aqCv99J>E8X2wlKinZgTHDvO92uZb< zBOKv8J_+Tnkd+Cv)=VYfw9b)@waAf^7+~qsa(mC@b_L0oR2{fFY|e;}mblL!1hG8vu7e^*OtWhE%@>{Rp!^GgJ*C{Qv2q$-k8QI} zQ`>b5(n146PltXcB)(3?1sDb5q_R>3)C=vSUOc`iJhMYgqwY!lkT-d2Q1us$aax1Z z799a-boaYFL-=va!+h5QhHY;@-UPC(EVctDQA!EreLv3YFIhT30B5q=tLrN+E>jl< z$M$@Q@J?VT!wYtz;5lp~;&|lg{@Bp1lIvHqY+aLhjJo|hrU2$e-o~}2(k7TBkEb0k zrPGBK&t3tPu>2b4aaIOhT*^K&1*LFRRN}?TPnKNE2T@g*1okr6^o^S;^uL|q>&n#S zO`7ym7VJ9_e{ox6YIrpTNJq}?)Xq%XZiGE=X9e_*)tZT=w1%Ui=Med{df-Vj``0?) z$ptRaw=ZL2YuaPx2g_N!M0WW6<=Ja?al$X@AhB?{JW1rX@aVXYg%UoOw+DrZky)8dr6M zbRy5hW903pC~E+&J@{_s4D?O^{#<#SMg~D7ePjQKLQ|1Ze;l2i7)^CSKW?cGYI`s+ z=uKzRJ}}ks)n_ci7XHK7gi@Wh>-pzlvnhCx-pwyMnxwq>sN2!Y2LD^kyY`c6{j@z2Q~&$z+c7EsYj70GLX?V^>3Z44|rwXW9dbUxDc zQ6mK$1UL9`8`-*g=5)PU%UE+g`rkLkv5XBn>DP=;i?@J`R|iNZ;>ffSGNuvXsY7_d zDj7Q&&;Bn_5J31*z{SPe$|len&YKs(3KcbcV$XyuOza!W?XVS(?(G>8j@OfWfymsQ z1jP`RvH9BB-k$0DsdLV|gniPajHk&5o3M=&dqkzfzXt`h42^SSpbi$<7MTOV8$(^g6R9{e>?an$cx#-+8-`VkfNdK_S6;-iQ+E9jnyKT3I_tmV-0-A%_}9 zf>b^H_K}&{cGDZ*M$pOgG=y#8(xEorOr=UH;(Uxu{hKs}q^9lE{P@`xUef0chofdI zOm4UMdxDXpByxV2Px?Z9uV%6MsTA-=dd!Ey*wQ=ADjwUSb#{s!C}m;Us+=gLhn>B<@GKu zdc|9#M`-9Fg8w({9X)$Am3;hqon7a&8pVo4-HTktq@b*xYBv%Ritj>Xl;#i!u1$g< zT7@w1V!bEUfZ>VNp8(n8L$++f{qb{q!pw{sg|Fv!A*n$vP+^STA1hm5FBM=Q;CKd1xXbaIt~uZ!eDdR^n)ev{CXge1xs&0#$RP;iS^wbM7NKZhk#uBhAYrf@Ef zmO75D)e0OnWwzpU2>Lj0t2ZYeqeQO0KiqURZbl~nB;%58n@ZBU=3ICY4Z?1&7Q_K_ z60&T)#Dqosx9!A^Birm^&j+pZA<|&MLeq8E#(c%{bWDKo4j2$VPPRa#QQ^=zSp!Q= zt^1M%spP|T{*Cc?2E^d(>K9>lR<+(Hu9nfIkV(ojS}ltnHSTXQ_MT{{&_$wk0T+yt^e&}%<)ZkJavIQAc zRo@o~Ivdp1>GH<~mopQ61L)T^>A3U!*rxb!1~z7NDEsQ&s6?Q!pT#+m9!gh~nL z*TLLEMW*U$>dOy%z7wJU^x$;68AIn8kW(9#sgZL|0s>jUY+B)^EPM$W_U%Lu@S&P; zR$B0kNxZYH&Oe1VFiJRHjlH7KEX_Uf%3K(e|w68L+{$wFF)C_v>rxH*=kmcz_;U z%@w?IlIO0U^d*&BY zRGKb0!7kM(DN{rQK9UY);=|F|QPae{U_+%I3JB0A5Zc@tY;lB|R`mJAZ&w-Z;^j5+9m2PJ>>NhgdJ3xLOnw+3Valwg2p#Bj%+MUz?{{!xMPR^2sn zf<->)DU^Nt* z?gbp@)cx6mrS18#z%wAbmRAPU6T-1&xeTedIwPrD?zaYHc01|*I-5P}&Aa2D^owW) z5vXj*bzJ48MpmyDRt$TB%RroL6}B`K|C{o(T{-r8#u??VPrg4U1g#r@KvfZPKVrlq zedY~31&=gdU#+x#Qr7l({eoRl)(^-cgM5v4bqD^II@@(G#G(1vGehJfjzC|yAFwe^ zW2Tu3W?U-mmZSikynW7yP#8el!&!oBu~;PZ7H*ol(I7qa)_3FK>k&@8#Q)70q_1oV zDvsrfC&2;vLv@{SR)wPTTQ67?oPK?-j!w{iS_W}~AH(*dwcCvrD=s8Kbl~e(ctQnG z4uJ>F4cq)sNMc;dpc0Y}jkYM%>*L=Z!Wqd+_$?^p_ku;zM`?6na076(WxK2v!HkxR zB;_r(L_mkS;!Hg#kRww)8F7s=HQD~xXwwPb%JF#dB$@FP{PYVSim&MP_V!%%R?VJi zNcE^z_n)Y@w$rKs0Q(>f$jfl5^+#@u*MNsM-|cPO-tCJ&-SmwDxE@4nbgP^ElsB2d zXy48XvzTl((8)iRwoRK_!O*Qpb8doBX-_9-Qfrv8>7d^#{w zGXdZjzjgA%d^uI#&D`iRX%s5x<%!)Tvi<~}%?;x$IW}xbCL@U|69A3d12hpv*c5ZC&u>oU&FQ^JlDbJ8#cBa=?4rLmXj6 z@(EQAW{co5S08(ikr3Y^n3?-yD?@;Mw^J1(nNG*Z;D|JoqSII-u`HJX=YmC9j?n44 zv*~&NV%}ePra|c4AI1(!73G^P=USW7>fdzQe^`Qx%#~vmGOHIVe8niaawu5#)HXW1#`?{H^ zXP{t95jopROK8RK`6Ga|?VCcK&}qy+NpWl0uNSd7fNT;<=mlba6Iw@TzZN3D!}qZc z@(P*#I|F^2J5dZdU2oNe0!G>H8Ho<8N&2_ncK8WJ^4eMI^~fNKiQ2mQ3HqH6c@iV% z%SfZ!IGZ)BJD|iRLK-&0w>$%)uYK$-z`?+Ge4)G*|DSAccy*9X6mX%sKh?^j(@H?g<+SioqX>g!q%6WObcY~0 zQ3zS$Z3^Vo)qF|fg4NeBz=5kQQb33tsxFfb5qSjni;QwrNB4ca_X+rjD_~;csr`p( zq_#sUfBXCTrGYqn)~}VQ+oRl=s~zw%>J3%Cd~2Eky*r-@V$j1pX*{>NKH!Xi zT7q_QxX^!WZ5tDFqL#!Dgp#~==Cj|VDM~V{QY?+9z4bSD`vd^U3@Tg16V)75tMh~H zMpV0z*iySc_Z@6Fu{Y8rK^eS4Kx`Q=)!`SV7^-=CY_^^==(9QP0WReBZoF2NLUI^; zp^gkl(r6gu`2B%skKp_Yg7i_GvdsTy763Erph;7_9q&SKa|YCgj5?iBn3#fzoY$Jb zovIYH$RstRhmlVkhHrNmwi`CCwz5ND0WgeUIO6A}reV{=VQ)WZB@UD2t%h?7pMa)L zZ??s?RVvd7`!+_`(qM* zdAN0gL(v*5A9RU>usjbZUt0Zl#*dUEjB!L;@kT|+g(Bu{l2O@qQcyJmeD+1#dpIzv zxFraVHvp3Z^TVGa+=rsc%o_hI-QQNa*eC@m=ikv!7^un!V~S zNxYaTx=boLtUbzo&4R*n^ZA9vRWWiuS8xa2I4Zb`aRsr-Db+)qN`66JB$_0Sb0npT zX?dx6@iAORml()T$M3fN!0JSfbGnxTW2t#RUaDD+sMOTgeqZS>3LW9nW;W(0%F)1N zQ6HnrnVk8;C1+|XX3c&Tz@n79^^h{j)EjiCC6?5u$7{d=AM_~X)ZO42WW}*1F-XWf zMX=y7C_eh>B+{y!5Vi^QbMP&2;l5Zp*5vS_MFg5vaQ{b(koK&~Uj9c5l29GxTR4Iv&PWk zP|ClDNToqivdm`CEgKCeUxep@fn1RNc?ME^i_{YjDU$3HL3RotT>jY~69Kl|LcYOy zvi%*wN}a*G-q^dyiP|Lm1XbXeqDeQ`e@~#tmGRoOraQ;k; zg?iilk`dSQgXvM57p9+|tIn}G5&C<$U1vIRf!7F%#UeEIlUV^r!#&u~zAuF{I|w8* zm@>2&CwO@?JcgDNy6eSr9+a;rEYS2NqX*k}numoj@Zk1E zE!vH`y|5Q1Kn9iEcG8fB5>G){%od!^JqWIos5XsM3S%34dveu<$mh)?Tt=J++OonY z``TXJKxhCY#7x8`0pcLWTKysw_`*vhG8ugc_&7fWQp#QQ!jA^F**+-AD7Bi*oRugT zt$NG6q2w(omZIWuc_}q&!Z@HK#bk)6d$PI=@PvhEV?l;g-j3n@ zDnI3Goz{VGbUlq{Nph4^ac22v0S>D_2ZnccS3k zL_PsPtaG#I+^5Bz?`c% zmFHD8n;0oH+lsr~>hj60ASs*k;2KU*nmD32HOP?@$eg8k*jmoH>88wPvLroSV9-1jE`=3M}T)p_4v`3CN0~q?p&s_rC!=TToV@ z=S#k{@+6a%68hCo%C5wU-QWG=cL;9<24(T$6d+8p0d$cv(sBZ)suOr z@)<=oF^XuCJ!y~Ll6RWxyZ-nPM)M65J0OY8id0$q5!Y-+>7F1wo)6&j#KmMZcZe|E2=}#+ zG)Eo^m|uD+5r(4{12ex&Iqkas^yDX|obNmSbJhzwQ5???uO!o&FlVM_QhK2)U&Sbc zjbj)*t^$c5zlFcvlAvL)nD50$V?sUN`frg*z{AN1LzB3ctz6mnll94I9$G&?65as2 z5!P$jE6~ZI&xvX&@-W+OdKl|0s*||g3ZL9yA_FU+yI^A`v-c6+&*ABJACH{{n~1;2 zX|314``!|ogBwHl=8REiFZ!T-2@nWc^}49%r{gGB9Ume&q5~`W9KDI3oZI>tcJc2g z`@hUq0CV+#Fq@G9F$(JcVh?}SL{&=odOPep6o9KSA;PpiAk$SWxu45mF+8PvJ)Psd z)O*AX82R-5M+X}b^$>@>!9|bj-TzRC8l}h}2HtsxJe95v7S11YbHHFV6BA#TbLpJ{ z8f}9YF$&se&bfckw`l4=Ji;Xk&x$Ju%s#CF-e*RMYU&5@L^4hfNha#C6=s%2o4`Q< z1>99@AO(_v>OT`SR`mXxNq~4G?FzU)=fp3oT^tr|WT~C~qBYKps32%{l14}0e8663 zRquSWhVqEq!^T&BvZdJ>D+hb`!$2H}1~2}(XmfA-SLn@4r3Gc;0neyb|J@sU2x6&% za?_8MPnE>!MC!EiNC?Kns#n%-$P4Y(&*^o1EDf5<*FhPY*V2->U(N}%vl>LSmJHm3 zpEFI&9V}mbvg$7e$Xi|GnC!O2BvT6ulA)!R$%fp&#Y3^JJXm?&5RasCbVbTNbKCl) z2>wEtk!kP`VT6LGwhye2FO6614dn>*eEc6J$_zFQaz6CJBgXUon@0fIz&I&eY0bab zTFTGNhq0YS693$~?b8Gp)(2lKbZZMvkkjUu-{dj-8^&+QmxQAIN=}839g(-A0%bu~f2}~pdxn*OvhN{EL z^Sp8;t0pB=E5M<1$Yy_f3i47#wq8MlS{@`)H9S$==U!NR$zyUvd|9Y<(O9{L@(_!K zAL2_j=I97GLm6MAUC!grcMH_P-CYQ4kw`S*qkqRd6;1%(8>apTRLi~p{yqXL%-+x= z$3NiXF*z_RFzTeiEumYh+XZ>Qv6wve4JvB@w-Z2nF&w^20A@AgwSRa$739R~@!q zwqO<=W_E!}jF7@A^m0-}3_vtnT)$g7=GnO2QMP-2%%P-3d@)-5G6pP4nTN{%J(>LP ziI?fyduKYyH`pS<3vqf397@gf)kLP2ab;lWQ=q>({iDg(=>`RDt# zl19+Ody%*74+?iyU=$yt%69oU<-{c?B$xP!bOKI^#UyF@Rf?i%V^OonO717gq#0l} z%P%M5l#z8G?>;>5JiD(uQL1q_CUGPa&G0m~o{zwmH|-()r%!TJgkNBQmxmd?F^$(h zApO%3u}e@qr|#K?GXu4!v$wlbREp|zU_*cos-+OA!((cb#(sv(pYO{pQD6p&)BVZC zZ)HtC)lvsmR0d_byi~0=4Jb|!qY(%)cTEY1N2yIRu>w|Ncs>gWMoa`2M&p7J*9~pb z&!XtXY0}wn@<-{h9dLNxQeD-=?`N&r!a@HbNHVn{(V6QS-~*o)`H%*6zvbcwgM(Q= zAWsK&{B#4ktq$)}qRgR;)jL+$qR7EXjL`y5Tb?YN6kbP1S7%)p-SL{ZbJ+s*c4SM9 zn=bDWXu*`+uJ)_b>d%KxlsIS?16oJ=;f8Hqk4cP?s^NA!=JoUarq4F=yh+h$SbY#02_GrTs>r%- zRlOZoXTpRk*cHl~w^!?-Ovt)-7YTORU$7Khk{~}Vms_rH{^gX>s#t?D9gtC6N?3k6 zs;zS>$q}1J`-F%a+KfZoOh##DxZHdh29Q9SbN>W)Dt;@K;= zhf{|C4+A@-k_T&bc5Hi`%dh*2q4$I>Qu)da&n8X*X}- zN~DIT$B!d;+}W@1m3;fY-rYlr9u7KV>aKaknd9(GM9Xeg7!w6;0~MYQu&r)RpoUGI zjt3jlWm>qIToey|jV|~LZVi=R$2!oi^qWV*u z(?WoeNDkIc`aMZO>Ue_P=%pK`{s{|k+Q2KEnhEp_*$yws1WKW9Dk8o_XdMm2yLs+^ zqV%!bmuGHt4nSAL`Mk~xM#|z!+v2ARFbts#?)j=M68dAb0r7p&g?NExR(ki-aqn6r zf3Qp|`l4X6iSvE8bLXP;dm%w8A~Dk1_NH7JKb#>n&wxZ7*W1>3pbbi|mQ+yRPGs<0 zbH=uOza#jeUB#h8otMaAhqF-6$#pR{RvONN=OQMT6{~($0kIQR&wGD`t6aF}+>Re3!BFj|=f{;#mz~g3lh$L%56)xKsL^EYk~AYtDmW~s z=-EoI;MpXjQ4#v194TDK=Z;Yq#5N}6?7!}e++7_Gh2s27WHy2C*T4G40&n|<=!0@3 zYh>Bm!z3f6vst|iP*LbnQke|@_HSf>`{|gxzlRV&NIhWV6#HEnJ5m%g(!Pa?)UAHMoK)4Z9wzEP(;1UzpT&phiV5- z?5ak}KOG&9A`rErB~zZr%x6*k%U&15ba$entg(~pz%VVQdj|Yt;-)iD_C;Ovythtw zKaFR-}O_&XUY%+*ffMl@u-7 z?q>y8_w>CTD-*r&)%p*!@02|b(r<%Wznt_0t%VZRDWV?-8ND-@9c}B-i_>K`fPHieVH5P9 ztN{tEk8PJUNh=(AmP!n^ak7c#`WeIWHtp34{Oq3*Ite^fNadexXMuX!YsqgCV$Ivh z7@)ml0iGez29-ySk4E%+w?B6r_tFyNF<|}`v8zqC5XfUmM!$L6Jl)Piv{V17<9muF zs85XJxxF>G6Or(^CKyV6qXH}V13i<=VIRu=)s@dViak50Hf(ntyy8M}f4vD+iITuf1jFM$~8!kA7Zr}IN6`N z5Gz+J#`s88P0tscDjf|K7dJJOSefLk0VTwXBLhc<{$-tPpnTb4;4^Yg(>;y_y$P)i ze>ufyLB>-gC|Fl7XGZ&jcTE~2A$Io&aXps5Z&ymyu|XZH!;waNt}`YrKa}1YDnaj( z#;x(mt-#VTs#pARu|;~W_eEBknKPuH#Fo_ET= zzwzT`$o?y5YRW3_=SNLr@XveeLlveFaWPRSEsQE`73L5LM7)F6TlBLaLW{Z>Gy*~< z5p)l+P|%u;j{rP<4_xqPxGXljij7u=^DK>K0|}&!zOTa*-y-z?M0&V~k+otQlA;Il zxPYORQ5$^l)XF@!4$u98@=)?6tJlK@dy^A`#mtPQ#8<6wX(EH8YZ2C-8WV;p3-!>Z zI7))p=u&OMye3z=)%COtvUVQc(Mb;qqV+ga&H_sUxod->3eMc$^l!!GwG${QrF5}+ zC~3px(8J|P*D*`j^Qifv#O3zpW1GapEdp#8G8d6#P5r23iN#C#NgRoL96zHq`$Bp+ zS}?Vb63kYd^;ph#+6G>R^$EV*B$R5*^gs|h*LZ3FjQ_(UvYLs$>`JjJ-?-aPINjII zz`4;+I*KTL_8#1iV2)x;e2gmJKiMsabALD$mX!Z&{hbZ8x-QM^{P~fdv4Vjw6_?Dc zd5|NmP3f0-mpKPjLsz`V^}*eBa1r+QalH<5t!GD-_8>q4*K;If>~~|F@P#Y zn=NW;J?k&^>U5G$ED;)pl#)m+OB}tII2#?4I)HzMc-=RUofx4^7oiLpv&l_IEe(#0 zlO&B^wE2zUYJOXw@O0Vx+IkJqREo5kuY>U(gP}MTVBWUiUJVP$RgBdN!_bYef#LV; z01%HrG5AusfMJ?6s%Givxv$0rB(ND)-iml@_YS{o=*%@gS{>m=p`1HrpAVrIeO` zfTUDIk_@hqT&Ki0atW<#FI@79pUmyY62Z(7Y%#_hhkZ7^&Ff3p{Ge3I87;OOn0NFba!;n*@zWk+kdwQOMVz}D=oz4o&msC~ErDZdP4P>PE&A}^vtzq}cEN^-GadrOMRoj@lQj=YN{GwT zZAD>GCCAyHkN9?yRLWYnZ~Ak?oN)6CW)d$gi{90CH{&?0@Uz~ivEJ|?`0~RYy9iOU z1kX?Y^Qfp4T;(n+e8gU+d;Bx0RRxEMpiHUp3KOkF0owV10>E{ z2b(wh4fXUJP8*`SsfgSOyoJM{u4U(8{jXcwmQ*k>xELXb?44MzfQ%*-OSWZCD!W$Ec;{e%Sc`wE)Si=M{l9TKUR5`%1^V~zE0xG3 ze-H~?8GyJqdj^ka%vJ`R&a`=IRBWk+&REUCSJ+j-Ny^TAO)8&^&wn!ctTk2l!HUO- z?=GLnoVXJ{R|kzTIX+Tk2~`cJ`Qg7OE_=E+5DDd>Cb;ZtwsW3L6cBuL5!T><=F)^k&EwV zn>q6rwQ416B(D$SND1(1>o5v@aH$K?I1AC^6hfe<_|f}_!o^bIZ1_ooB8m7@U!x~> z!|9;)$`WhLK=lz#Z5A~j6-dM(8KDJUc#RL&U+k{tKdtmIBV8>wlWw%tqxz^;3tL~! ziJaW+uT<#;WS5p=5O6wXwzLFhXOjd=Adv{s`zPTNoBhaXp)NuHju(mf9X?Rv7e6XW zBPn7^?#8+xjALhaS@?Ge8Gc=y*Xpn3G7+(<@O{^a{76YV1f4ZA;rQVs*GOzy&&-Ph z-%(HTJG6Xm>lC77oBE*S;HA4Jw-Bf}a36$Tl=8ZW_Y2&oU=u-E)3@aNRMuaiXL#pe z2dLT|f~^%qXzSVFkx5PP1xXfB77l{U?0s3=;MkT&ak^~}{jfjwR2i&%wpwmpbX)+v zNchwisl>551JxNt4VxY_Uw*XmHc#tm2S+v~(=v(ZB)Zypg`C;IE?9F`w3rmJU<*<~ zALwAxXi{=9{-P`jE%b}06*Fv(q9?YBw&4;!YDB@ur(g3}!_sZzBL;)^TFj?v2Jziy z@o}q2pQR!)E7ABtC#}PNKmOJ?X}Zme?e6+GzQZVX)wdM|A!z%XYSph_f}y^uQj4YC zEYvnb)cAUwb*itnl^}Keh0(tX`37($*CykfVc^zkwftU3ndS{BL}CBU>{5gmEY46| z&mQ+E=O_04iOwR*7QCx~>Zl~E+vZaxg$dK(h*q~JD0fAyVCiZ{!{Ty)ui19N01{wOO67}8^A%4{N96q3fx(Fb6_&1kBa8a#JY@IlxymUzDO z>$pSv(yI|znSIo~x4s%77E0ffrd0ex{3PE)OPNbX7rBL?N*6z_=hHVkJQ`&$lNGLg zmJ3v<{=CE%AVZ`H(SQ@PIx&H+z%(m3nxK;^l*nZ9TXoEW$VuARP&C;Wtp$FP6b_wD zsH(~gyE9KiCFZ9Vj8cT1PG`u+nri4(ifzO}TlR&p>H2!MBUbc8;z9V;qpA>y*G1Pw z`-G_9u2JWHLKFW%gC_oJT%z#x_**f`q&xiFNd9a9X?Qv_M5(yB)qa7^!O?qpa0}hC zJq^VlXeL9PJ32_kkvph4ph|wRSr&2$7%Y9iwsZVHeaN$;<{f#hWggJv%PF3jK6Cs? z&a0^SzI@-XrTnRR#bW?Sg9C)m?BSph#B{ef99OyZFZNwOLF6plv}ma5vIZ|*HY{U0nC09JDO(;>K9t>in4#%~u7`|q0(sI1f zs5T@;j24Lv^1O^Vo#L3hw7xiOl)*G#W`jo%u3+!51lkz?&E3tBsKp%eCgXh`V)5lS zgAba*477p^sZA_?-4#ZH|ITCqZm`#ZclPxRtjLJv9oBZpXz`q)1cakU-y0Z=~{9U{t2YqLhmaRSCEb_o-)bNn~52EUs`0ZG$ z2(cRuPY5EJ8ttssg2dsn;RL(F*4ny2tX3-=HimXzjc5&RI$%bv z2kH74b@-oIy|`x_s3XkgU0R3x()9Uj}lMsyGCr9m=7x%tB zJRlMKg4|sva4B3kA6R<#QMMX6st~FkSAS6=c=s$nNclHnlz8+cxk%{bwE*>HZ6`|y z5{t*bS$hiwOGD-95Oj88a_WeCeadlr#;cSBZ(xHfqizYoH!*h9s`dW zo;CW1BUt{CYu~ad!Dt#d^%Yd{(w<)>?P5fSzB9=+(3CRa&~JUq+R=)N%CzR#xId@` z$DHFrLE*N%{9OPJ@&<%0(F%zvG)hU0;9s8zeg(T>9NR*lmWzCMm_;dYwt}dQyTv^= z=Yhu)$w8Ix5>JOW`gC7^_|`oQ@`fHOBubQ&F&d%_dX|5pp1)rm$m4J|IKgHwwBF?@ z6+WBPEoTxe`0OFfR-#o}aO?J|32wm(mcCX%R6x&_PryW)k#OBj*>$B8{ptGb-{h<6 z1r=6g8dHeiZ3cG`>8&S9+hr#9C6Qgwno77xI`gX2zFB9?4>c|57dG*{<&UuVPa1e8 zRwU*gB9vE1FjLy7!DS?=DiJlsQI44AZ`UYe={l5t%)C~t6Kv38R4f-IQ-qO*v$i1! zvTlXmNAG0`sr;lFpynng_1A1)F+2_4na{mgn(l--_N`RfWrLo=u|RLafBZ z3=OcD_G1Ki{v3=Lvss2Rn$JG>fd)UAo2*-+#=U%dnERFno;Wf*tZ?3cwhpJ!fxuif zWcK$fv;|mcM2$fS=kZJbGrVH@H8I5me57_!ia+-6n>dkz^Xf#7M<-XMo+1Q8gQNk#Hi0<4Qg;%TqqbfugpMTN$ zmdMocQdhXs;LoJ^Z;BT_&I>luvo9AywkUn~rHXCzKTGq8L=)Vjt{6cDeGJe$to>>! zD}TpdU7a!E@E(k!adr7$0zosvEM`VjyBgu$Bf-R9#4kFmv6AFPMMWLV>O3k9#nk;T z4+>*Q&t~jSL=1noPfvqAoJCf!=zFo&;EnABN?_SO+SEcv_)qa4wtMI~x#fzKnvq5N zMuko2C8U?FldTF`XEDsiiUX6a@L8z72(pGE3VdvfJ|9u{wwH#=zi-a`k)sP~L}vV* zWPQNdxu7KT>(P#6K}lDzo~V>^ysG{5jyG?^^=F{QE}5Jk8{wqvs;CAT+JISBQ#ll3 zZ0M@sfSBWAg!AaDZZ`t|sagh0lbt2fv4aIVgNEKVc0#tr~wE`oVwE zc~W6aBE=1>1Es4fGMIVJZ)x#Wx8f$#%Y*H+`QC>-TLr^v)vnTprFSl@ZvaDr^M*qD z@F8%XY+|vg+c;36Q<6+1N9#4SesQf>YkXem^)a-sBt)Rlni$iOKh9p@y@SYmho7BC zs7Hu~-+8D^=SN~g&=&ZO`<#58+{M=NN8v0U(W~=cCzYNyF~+hXV9Ob(FG$h(5Vs0p z$95~i^5=;+vt~jMP_|RiQp{yxpDv3;fzBhk=6&YCmqQY7!*5@A1{u71-k`MfpKW2w zD8njEKK2!Wl_P~{J z+$C?EQ@}dC%oq$w(^uI+jPNkDB|+nRHca!o)c>cI^Neb$*&48dfOHY0cL85OLO|(V zLY1O|5CYPIbfg!7h!p7%L3)=K!O*3J8YuzkAYBO%X@XR#5%`YpcmLe~cdh)`>+C6e z&fY6CXJ+=(7imc8Mr5wDSCC{-Z>p5ppVF@|{t@tyWC}t5s7yo3VuKd)V3D1y>CK9O z!l6L>Wl}ehGiK5)r9?${b=)3XC?wv=#Z8dNRS}I!` z?>pUc*PiaWnMk)-U&NU+K!~HcH@v_zPy}oIi6|yqL4tq zduL{g$1vXUTo#~BNxngH?K)YFaY@Tur7>n@ib`7$gNCJ+qcK~ty4h43sj45oA0aDY zj|JF2lh~TNhU=P$M3{)aXT}UMYhULBGI?)OaKgcfT=U*?Ylvecfo&w-oj@EsIn{o= zl_`gV7Y}v?owlj8S}@C8-{Xwifqh%T=Iy{Zr?I|W<3`{1jnJUZ-TUQmExL8TnAl?k z{Y?qia!QzqwfVe+n9KVT+ANa?<$_d(48lux6njAm+CDsY4j0sx9TJz1bFn(~%Rj;# z>qb4G@_s*LHC>{#a_C+oX}Qq89$XZJUP^XzN|RjN;v3Us<;y2tAE(tttiIG%KNk)U zm)mY4IJFbMy8xN#vkF95s2N2mM@EZ3&r+0%V(0ev#_>7%z&6kzSwtI8(#FN>!4_11 zuCo5ObmsnqI1t~xKVPv|f~QQnfV!oK(jzHL>RnBz;miR9PBy-Fty4P6wRc^YPc!`>U~8>6PJ+*~x{P>Kpi_DC^Ld++A1A6CNm zT9?$96+I9tS(^HS)k8xwsczL6t~zI~yH-~RU^cq*-!<&!&^=u1f=5^d0r>^~mp`xU zRWde_wgqh5m8QAMqW5V)nmy@eg)ci;=%xs~HG@^_v-kMy@gsg`-9zWkD1?u)sdrI+d$)6xZl|ke>?&!g zLjQdwko$4EPpi@SWwp9ADlmF z<9Q%`uXUKM!W{;$AxFm~qUgm~H*Gx%R~LHub_o$fV%Ijsd1r#j><^yLthj+K24vHp z3)?uJoK5Gic;}msBy!!upI3FFNvB}Lc}fa4)>*s-n~Q3%h`*IDPWuJF`RVYai>a`@ zDN10zP@>>;DN^mm)(Hg;`)ZL)_9x{gCq#O z>o$i)S6Kr140+p+ri2P;==6Bly$V7jq}W6j(bkPu(b)Q>UVL%FZn@^Pt`gTzKK2`( zAWi$Gmu6%U!-~5p&)w?@-rl`m$#dJ)>CNFjg_>1-3(%euUGg62|}r)SE5Z~XejB?GRI;G8?a2m7uC1>2Skut*;zZJdH(ox@1x zgV6q6@~705I}CBSn35;mrsx8jU$A7CX$E(54cAkz)<|;ZGaq^eGwZ44qgT-(XNKL0 z+})QG98Ej^p8*&4(aKbKQ|2y>p&0e?niP!9P+X6!!Eb+~dwwhIO#vJir32B3EkaX2 zQ{A3io%dRmqjRm|*}9b>k(+8z-t%Z{)|2>Y#fwkoa%e(Ut_%N+t2M*FCG1S#dbqqU~y0+Ke=lR%&rUzrED%k(L|&{k2X!Xi3?O zm4^8|AUV2nCC;kV19tq#dMc*9U75H1H;#vV=D8r;YBvy4TK3Gh^ZLuWmux^$JaIKVd}$Yc zHB%27@XGO@K;=v5@3VCb_*2)_&07!ZWi7PYkztLBZwI=XWIt$;d=qaDQQQ_T)^OHf zePAea)XgK|-WH2kL+(DN7rF}BEl|hsaxpAwZMwn_whTb zK@dY~{dZUZa8SkaF%ujs6E}A6MVtSj?H05UYhuK0rE1+So8X zQw|(fN$XFsQjyF`?D_VXk1L@RM@RVMqE*^~qbKaY9zlLxD`%+cgP`j(7U+RRg4*^d zPhXC#?#zTft6jBh#FRCc_?dNMkyn$ZhBgaP*Tj>k17LED&Rq;KYl zw|c~Dai4M#kMU}7`15S`TN%FEShEmxynGsW{SFjcrb5OLsAMY1M3DULe@`g#ufv?}+rszM+;p1fO4g^hk zOEg=2lIw%H-VK%Fd;D>Pb;_4FiGN{gI61D;uD>V!oyBY-2NJk$LxeUA0;$6n&f~{# zcQ!tDqK&8wGAUienk_akzXOWa_LIE!exE?Iak{86t@dHN&r9?r;{~|^UxKy@K$aaK z+VkC`JzdKt?8}ePVJ#feUWSYlvL|^6^cxj{pYnf5dnIlhz`kKrOCPZ$nGaRj# z{aUWy7=P@$5se9OlkeyXLVJMd=EpZ)B#MtIXiELxt9dy5pIgqYkyL1o+ zv#^k%`er_lrldU8Z=gg6dzJG3F&E~|K} zGnSkB3Gjsu@*tZq67SVss$upyA+aYc`wFdhvgSbaYvV)k6^;>2t91Ic;9ZuTT?3)E z$9(3BRGB`Vw7oo8ih$&@W{hpYD7p$3v2{@ImWk-p<4 z+umx4fJsB$<{IiGPTXMQaEC7l=5JdD@6 z(?Eig4ct{VG4D@w{r=2TUbx`#-lr>~TIY=)v8!W%LlIO444I-mzyP!7KzQu4c2C#V zz|ESeQl$}LN`l?^9R`1=BXTIWPpu*%IYlnc4H{`yOOA3PyPR&$@v#O#Y~skdif7>! zZPq!R#wulO9-I{T<2^+r?{%geeo2=?r@{5U!L+o~%;CYET~L)f$PCWiH;SL^u^!R- z5pO766Hgl&MNkE$clo|ADhhYVdl?Df@Dg3B+WagRh`}GSI=|JxDcs!L09;KQ^|^^I z09BdXT4>7_GjE6|vI7{t9@~@lszs)xtQA?hy?{@QFE_?xEafmM#UDS$0jpTHprb9m zRNcK2aYRfU2;(@su=(Iatl`1-_6s0|iE_K0_~Oc|{df5ulTS^7&7>xvcae5OLqqz; zZ(h_12??$<)uE{TSAo6Lz|CF6*0<)mLgJ6`y7%wjKPu*kv{~}-XGjV>?7NM$u2?BU z<=Fz5M_|7@_r8n_%ka-iil;S^r{O$*Uc+&TdjR{7WOu7>WEle@9{qwlI!5DoLRt%n z0Pv6tPSfJ|YZ-v{1z#NGucew=W_tL@&iZcnWE~y((eWq`W=RpI*%NwT=B%FV@Lo4# zG5#m?`!ADKzT*xQgYmfIb6B<;$3Zwoa60vMczH!QGT&Bqsuw@A&zj z80~ZL9@IGjtgGH$O?Gy6=L-7Oi1Ot|S)cVA+0t%VV@d#|E^_Uq>$j$w-I=KhbGhJ? z-T12c)|dAv=(%%_f<)Ey6=H|7lI3$m_qTlW!R!8QKvDf#c+gdXi-_7_1JhT@^py_&KRL@^q zfUakSQyEp}ttIIRIk~zDWa=#wFAAw2en{?YU5exg*eFS-!c(ZJ%I6BvsDB0v1SC4mvoxyXXJ1 z3SuHdMGvC>_Dc_{f*s2 zSP9^Y%r8-P?dk4&p(JODY#O0vn}7czZmm4$?YXUk@YDMcXn2ucWuGFdQz17A>A=lx~q zzUu8kOaTL`Z}|~5FovP~rSnXrtEH4Uy)38KBz2W36b=_2c4R1Rw~r++{-cLuRxz28 zrFNT-PK_(hJY&D+?MuFe$*s3GW%l}H+v-hQ$V>o>e~G`f%A)?OmgZqR;rZ~7%cYgZ zRF`s-;y`0{wmw&p7o`&4OIatRribn`WyD2?85PSL8vzQ@|5t!z^kwbm^wdL}uM5j+ ztm17q$e%3e!7=(0tA&82*-q<@io%@D=|lE4Q&V39W3;6PLfTiS1(ilb<+|!EC)MuJ z<_d~3iqaVrR4W)lN;SW$I_HrXi4-j|Wy<@KmWfj7WwHaabN5%;;upf;=YfDvZ^w?f zlkFn%p?6E)qQ!v%Y-FgrCMy)VYwERA&H^R2z$30HSncfHW(&GU9=eF~sTaJG;!GKm zElfF2+?ie2qT>rYQ<~o|rod2=H-w7-xdf04C2wTDCNWx#l8Cebp%zohizIK9y#*em zhZ&WJeo?h!XUeDr$A%j@OnwP|#1=gW=$K8EWexuQ3<$}eWAiz#}RA)+$23D>GFd;4)7zJv1@tRa^ z9u;6Je+i(|$C}Cgl~J8cq2r`C_TQH+8nZ=fGmw>P1F`c(71H~@lZkD>mf9dSG+IwL zR9G?zOhfIGbI6<{LduYN6DTM6Q&9FPIlrVEm0nWWhn~On`_PZ9WA(JeuKjJnK9$7C zPN-b$-$D*d8T@rNeBnmVdHD|>0L^8uivFwE0TkXgps>FR>c9P@+q3`eqKqAAwmA(S zZRF%pP6H_x5e93lK2(;ymq}y8nn?B6EMqCM(!O9(dIMFU?xbO;aP6agEZeov2TzL6 zfYK0%2-x1*84H0xib<&S7#NbM{`vQ5K_?RfLmA+V|CvIi|G=42oS#2TGlQR>|JGyc z*yw0u)lgwz_Pa_}8Xc`o28bGx#K)Yly=o3yc1&d>ZXYPK9-7DP2zex8Ni(y9Y+|k5VLJTxMnAvoi zW?Gq^Hmu34y8c7fBRjKQ=?!(ib}9sokcUs(&bzKDND$7auh?!su(4EW*eJjiTJT>F`M)jHGLXkO9!sj6 z7eite4R{)5`jQ>U-7i_W#lP$kkeu>fwCX<0^y|4doo^V_qH9e$`9`O4^h+<@-7@j( zDRH5QI#}at%k(?-;nYo_`t8v%iW=t7?M^++^&@`}c2q$CTUi2P_Q6Ao3>Nc%ri1XM z@+D$uAfHrBHjM2eXmEZBVCafj-X+_EEgin;U;-U{XIA{#{s^Dmy4T7nGfu<_NP-eC zt0a)O8no=qOw6*YO4_H)to~Ga!6n+`Mltvsqn8`j+ZXR&U|haD-@xY)4xzXpC$7~H zt`ivC2{VKHbdOWx-3~w7-nwyiV=Nrmi(@Xd<$A=c5kSY@EYHXCd&cN0(3jwsFJEE} zFVi*jVu#NM@$FyK#T^5N!tOR)MZHwOpuBd3}c<&k`u8O)ny^4 zNwU3OOQ#*LNGJWN?Y=u=PpTJ5%9nHI&zP{|nY8+M3&W_`SFre#c)__cC;T`wH4p)n z-cks5@Az`3M`aI#!OA^p{eZjvaQF>4wL`-HLCe=WdY=vuCl2O3?c>Lt51CF*{4a|B z5f=aHG$mOqBEUS^r3wp=MNuYWr#01c4ycg&U^@SKjz6?pg844 z3IZ_+V;x|%XU^rh)fA`>kBQA|$E4KcYuPZ0J`8$_t=f$mXO z?rE40c9Wlq?i)zpukF`~44FQz=(S|2&a0-6-X{}5>fm$p4`nfj92GkWa-7?M$sxE{ zjM}r-R8wTxsoxGEov^KGH_zwditr%*;6L8z#Bz#Z+z7MHADZCEB+z- zk4$H^I@MwJw*Wbtt_QFhGkD>UjVj6M5rxt%bY|V4C&Z#vz$ya{B3i*^5oh;RcH@nj zPQX>lTzOFX4p|N&C+9mx!s7B?1e@NT5;!+j%&>t$C73@NO%K0sFS#M+g6<-m?3#wH zOfeL10Xnc61UzbCtTj6Ges-k(_mu70eJ1TW#Uoy`89PI@zLcBqAQIxAG6R_r!=(ci zDTh(3J+-6kgR~0NDjSG790uc*UQm2P5BFB_4i47#uTjMq%0%SZrG*y=165c2-`bI5 zH93d8USJ}M-6Y7t+>M6)F2|S-il<1S}11~{6Wr86eA#N%4c;h_s z1ONy#fl@3xH+cr!Dyid6I%Q=%2z*`~`DF_{81 zS-~w;U={hg=SW8r@OndG64ccVBE(sY>C8Z45GA1;y|?tk2CMJqqBNehb+#ZeGG)qw ziQ7OeV)EHslo3@gUgI~mq8$Pyj*Pb)=bE6t3Yk&dQW~LFL{=_22LOze_#0j%WEloe zr$y`)=p;osh*+?%NP z-$`TP9L87^Qf-hX72b)6=~GJh)Xa(bJ=DbtWdRY(_RDaFV*+i)7{{tJfK|AU4O%6F z_3=kPQt`=ti^&P6G2#2{fbHh7P1C1H`Oe%pLC}3&G^szcg*jz)2=3|iV8|M6t^xNP z4Gtg`Yf4|U*h-uIc1zJ)jMsDW5D95m9?Gfip?84Qz^~{+iAOc%NKA&vbaJ*`ch}nA zX}~{%CvcU&d8%+=Rgxvi;sH%kCaQy@=j}+vk^>eMGcSQAi5N+$Zs87t)%zLFnwUUz zm8i;v(4fg7{cFjeW?H7fZtb5_6KC79AgDjUA z?@LDp{I&nvwcNSN!dy@_=p1;Q;p9LnX7?{nbtswSjFT;JP|F8otKi?JIRegpY4bD zffOKT5R>1?u^ZYSH84u=_9+G^-=Zg|a!iwf7}>C&f7Zh+?){Y8N-CCMJro_RsE6)v z(`D_%oV3#4@rqu+D9a;n2Cx_rm(q{{`cRoIdV({4g1exM`S4>|yfYCQkfQ$wu06BOeEV|!@UeNPv_Okpsa}qyMij4O+3TXFy4{<@6lLgK|D{rj z+G$t0DS7LI3sb-2qdykbKfyi!<|qLcJj5gq;5nfeG66IA#bG4p%Wt=Qjt03>9uiUF^bR3P(sS$;u*);7L!RURTF4TD0-nX)Iaik8gh4; zh{qC&9vR6&uJd|4{$J4L-*u>YM3_tuf@uCrSC=@SyS&_zoucE}mUpqjx#q|Bk0@vE zIz7BAvvu?4_Iuiwei4Dt&lwYzN(Nbp{EZ_=j&u&!2esiky%-(gN8vZ)Qd|(DSF0S@n6S$ zh};3^wSIYRpF}!LY5Hk#q_Otu>z$ffT0_kdTJy|m|Dor81&JF&1%$0lkKBV8;(LsX zJ-SUF-o6XwbrVQ$d3uGi)I6i+xnAy4gP(U6c|nXOq3BL?8~>sJ3T<69I%m20N%u^4 z@!tBRE_GNM7zgzPodpSDun~g@myvq#mNjRWv6zu)thKxt<{s52&$bY#N9ZDFj z6Jhhkr;o6aFu~yzVc{39c-Y+9b-2%tXA*I?Oj3Vw#=J|P$u|-3;P)wC^bi-GWg`2! z?Oy#3_tlKONH=ChlSIVFGC!58Q7h9+;$ed94i_*N|My^L3FTM^VKqkZOi+^Qpwa!^ zewl1&ibIv#hhN@(PtV7+OlUzz-MVWbiF>4!C1UTeCd-crjh%=NiX zP&EldzG84qI6*J zF4@!wwr%)w!VC%;0JOFj9KfOcNe@3urCpKjxcr zAM$-5_-`TT(M!~G>KTC6dhF5(uT{ypIBWE%*>t_> z(7R#Fdt?LwA`5AkGT&<=#uE#htn=#wQA&-)22siP-JS+Xgq>cG+0L2XI~OM#<^c`a zF!UZe!EPM)M75PnJmf;MKaT8(F$$9k<7WpDAnr~U?b~7)!5hCy&VAKjapqR;W6LZ0 zg2s<~;6S>SE2fRZo(#5^l;9F{(a_hZuAZoqjo4C`VHJvyF;p_FC=kIJU)VI~6a%jB zuR4&MwNSV!H7hhukm9hnt#*AQpB+ncK}9rl3>Ie51=^gdfGlINhsC4mFqy0CA{#XBWC;P9#|t9=gprr4bh83)UL-{N4@t1S>=%9A97zx zBwf4V*Am(oF`MXk|MkzFtC|H~_!HDt`$n7G#r1TWsAanpZC2GW>wE#dN zUQTT@-VZO_mar$r4twt0{lvr6rQ2C4jbP!2lXkROa;y{w&f>x+hZyYWAx~S7a&i@l zzEn<~n|nT{E2$b?xkm;w;imfIPA*hL>P6RZdY7G80&PG6b{kZ3aHH)3|kEkVslELL_I57(O3fYuDdETaLM~W5L z=qj80Rr|t>p?Z|Zg%0_zIhG?qRO+F(&?P$kCK+oq<_Ud$BWj~Yv%O|%^5n91wjP7u zz@H2pUKZJ6@2``!=BQ>y&gES z`iNS-sAi-0is196?&C=MIKezV+4i&Q73xkC1)g}A3!ZScej%LJE!UuDYyuPV6 z*gbxPN=d4YeSxP|4@ZYSRy=!{Z5@A+QEmRr-tJOE@5WrNOx1#~=kpUJsoL``aY&5e z-aorGu#^8jVy>2VciVKOwNNlhn)WlELz~I^Cz?*4ct@o`PbQtXzjhlipSapNdWR4Rj?m;FXNoJ_!U}4ezUOGi>c18D5 zH0JXU@J>H-}13Fny-V>kTA_vj)hUtF-Ght2)Perz~5+2XD(9>U^oy zF4zQ^QdL5L@tWrX(@Vi_6=$Tt=0ATkc(Ns(6rev*^o5spVQXtYmCGQ4<(!KYieq1Y zSEu&KSY@2{Jlbwucnk2luAbqM@I^m8i~WX`Tk@KoNb9%=i4-nsJFh6y4kIRMIjU(! t*$*TwekH{r$~vAV{D-*&cVo?joXEbh`+`*qlRCe(?V3e*io4R5Jho literal 0 HcmV?d00001 diff --git a/_images/markov_prop_7_0.png b/_images/markov_prop_7_0.png new file mode 100644 index 0000000000000000000000000000000000000000..70dc5d524110f48fa9fdcd6ee58e5a4617df67b5 GIT binary patch literal 62852 zcmcG$bySp5A1+D^El7iOOM^&Q&RXZ6;~E9WVP?PY{>Ae=zh@hzrXq)pNrs7lfPk$aFRg)qfH(?%YoVipXUdPB zd<0)0uCEoe(7`WXbjwKacMKS=r$4mwpK5-t&3JE7Tz~ zg3B{vGmQw=2s@f#NZd2au5U_jd~YsAy2EDND4(E)^qO)<_t#z^kUoB|hsE7%$ql~k zM^WN__V|nwB7xF>US!k*Ndd1VeM0iTUSb}#qd$)E_zwJJIm;oVegEe_`hW25|Ce4u zQvfzn|3>jk&nws<2>G;QsqVaq3lTY@;j zG!ld|DUmJ|pT8Bq;U+Z>oN?>?ekzl1$vuysRZ5#NB00nBM1cldpdH}>@79VFSul+r zY>Lh@af%BGiee6_@2c-$^WCnw-V17Ro{rR0#{D>*{ zyyzzq9@k->d5*Rk`ZUG&Mm5`*SIN)ktZ(9exd?r2FrsN-?}_Zy69GG1<#iC6?B1U|l#c9jYd(SOrCU9d zY2%s4<+RrTCqeBee|(L8Bo=SR2oXAC5ajZElnT%LFSoa%bV=xa>wls#Q&RX8={%~J zcVcKz8mGdbI5AkG-6n1x0*Xup;xj5+U4M8hp zj;`;nsE}t$No2s#)YRPF-w&y%Ko5Hz9w2dX#reRp5c4)#G z_y`*(E&JBGu!R>dUxuu#7=NsYE|yCuDl9BCZMO~W-8n6+ug5z*J^ff)D+h&Q@hVHP zrUf%7fNdZpBYO>Q2@NhcgqJLiju5l@BXd32?qYth-JY0bMvC4X4W!^lclY)rnF;&q zEry(~|86G-iOfmVn*9}ZM5n*=Z}`EE9|ZxA(z3E7l`>lN*OGxreCL)zuLYo(m)#T%NVwtoS6yHVhy{yxh$@)pE~O> z9BI;EC9+f}oSK|8bs7uDC_#1scl@g$N1WMd^4hN!zB1O<0!lX{0%cADyId=`?cOVeusjOizN`A)`dr{7jhV^MmaI;v`kPtL0untj;%j>7R zG2m@bQL3TGm^ob3p8OG3(#}0k2ZEMOrL{G+C;3l#MRZ{yt&)eU4%F2TlATSYSU%q@vN=h2YKB7j5a(8zZ&ZoE&TLYOt)=0Oq$kkFO z1nbqCdCWo5tWwEj2Ku=o3apS#K_wLxb9Z;*rBCnrqX?cGp$w-Q+*GB{HJGAEO7N1& z@RIRSL77r^4-dh&VGIhHIXTiyDK9=Trh`jcj)ssXO@w`twc&bQe#F^in4w?%gbS4Y z`h!Upx}Ayqq2ZM0Bbb)&^YR8~X9=^ivz<5N1p0PPTm284x00)g@k3`87o+FrbvB)o zUuMZOaP#wrTk>GQ7TzJprduct70ZN=F*-3};ps`*YJ>w41y@ZYwM#?2;sZ1`8&2}Cg?%}! zJ{jDGSqhQD56|l~2+A20c8`uwCr$-@ued=eYHM$2!!O;APe!p#8CPffi5aFj&WyoL zXIEUV^{uITspxFWvl*Wg+|EBpV59i@%Xbl!>+e%PG#Lu#4&LA0)Z5H3Z3@|?RvGG4 z2>$a9+!TZ$A%X|5fB<XIeJ=M)ZE9N9U`v_?(P&nm$tf=SC zc&X@K+~-Wux6Tb7i;Ih~L$0k93<|I5l6rpreEdd~N)86gW$F3&vw7DbHY3OIi#8OJ z3!?OAA0$N`FAe08*SaPl3MihWR8+D)J|Zf4qWT)vo%UutlEz@Gp9vKw$wORSUB7qw z2qX_PfGjt!Cq+ejti9@~d3qn%z@HZmb#M~dRaf$V^+$f3=*5S*Qeh* zyj=BJ=!{|gmV#34JJmwPADG4JWy?xSV~ft_5sD3d#TsNFA216nhwXS>`Mx4HZ9+5+ z9(F>Zyv|Z$0u{~T+c}#IJF%3bBbTNAU=qC?rfpYxk;eL|Eys`T9=1&yLacv*p~AR z+CHz0WBR(SRyhH#w3t~8d-*LdaSUym;xm*38@tpl$;xKaMRkX^81@l#2oh*xRqZiD zphR)6Gi5Yn?hZG^c6GfnYIPalm-(R4d*+hLPN4KQ-F3s`+cQQ)S@^g$H>otpa`ut0 zmOO0urqY^VRhE_zuoS_fe+j^3vTVDmBflJY8fvfg!JMnDbH{V3r8m<;lbhXR3*If*$qQZU5Dm5 z>rCh;LF3F(p?E1~!q=d=Oitp)QwSmT7^V7yWe;28BjDmhDBBf}fDn__)}GXy8*OUe-L$hMcF#TTk)GMJA)Tic3d(BE7Q$o z_81jjvlI7&4{&MCNqVV>MmqPq(HoXLm!rD>L3v*<8`tjQ{1K^L^6GQ+ksc_ zd3U}*kBi>2;=wo1Eud(Cv4$NJzvkB4)L+glMrmrPoBE0@uBfJ_rU>1sGEDz51gyZd z2i3`TjZaKuxD^g=I>S9YqzGj~U(-?u!#^Ip@MKX;SHhoDIxr?s;;JV`nZXf$+w|D; za<&)@U1g;T6NgLeI20WRs>Ae;GWFY_hu{qnndclT3Cp!GLEZei?ehJzBQ+Tr*`Gzb zT5Vq9TK!C=Kj}|l3mIZz7JPUzQ=gc{_FgKIlw4l>q9Dp{HI@(du}8Sfz&1_<4Gauo zc>=fI()UALDK!+zF*z>gG7?l z*8ZTu0x|=ov9VEB{hukfH72*gUXHry-mffoqIP8u{fPmROw$_r$6o*Ud22;QIA4hk z8A#>F&^%oF-2R(RgOd%m5g(r%H((ALimZi2vHBG(-D2mjsN)O(%2Szw_{QyU|DQ^KjQpMLTy+wnCn zghI@laNX(qoWIUOR&)O#crkudD4voe(@Oz9rIa5!N$|GU1E z&7qODHP-jIx(#1o{MiFNJqX}}odGUw5c%*uu6NPQ!rz|?bO~z*2Y#1Jd1cMT-1eWu zK_q6u`XV{e(@S2GxBlG<(h6~lF3fV=hXpe*uuc|kTMiIxaj!%F-d(N^1tFt-Gk&S3 zr}x;LKqA^z#)P-Xv*C>@!YUn5QXprKU>Dz0O5Tx1mjlK_~>&E<2E-Rt4EQ}!6I45w zs=08CUu3;3GNF*8)CMg8C;^QqY;Ao8gTaVA{_5@de$z-Vhe4cU-hX^azEq?0!Lr$NtY`^6crSDrn?mRkv|f6L>k{LkmX^YC%dOfQM!zdkMNrNf?hAE@Y&CMP7{{8!4!X%9w;(oSwZQy4C`#2z- z+&njE0C1ZEhaLdy;22B&a0Z3{k|+7H8Cef(x)1KXGRFAKwHT!fgUDLlI(pb=tqb$> zZ$$G(wjYxvEDS|c-CGzSilQ^)o1bjpbLE_~p?eJ8S*2zGwt>>|y~|HTB}Y)5yH$tP zc*{+<(Oi{3TL!dmunHUs!8OvL&%}1a(++%XdXT*uT7~4KTb1A#zzYVBj<);kkeZV? zMqU0~`=X# z!2|W!Xpipq7v<}J`?chVVh*qJo2|{#kvsW`qNKTngrY#iyf@IABC`DYLC)nOU30R7 zIn$Wz>6$Vg+8Qfok)FCYT~UJgy|x73qt#|m7@BXjU=2#b>~XC zVm`z^mq&b|XBN~qq@EPU7)#oZXJ-fTsGMkmCOqrGFc;5gn)}GDad9P${ zYl}%SwEi`Tz6M4eI#DC5CrGWWRdFD*4742082vv5@W!}0GxoNilb$qQoX?e+52bR z6ljqs-#Fi@suLF9Qd2T27#1bTfB1m1!Pk4X=oxr!V>bO|(|LAbA>-%x5t+-s8e{)! ztDIMUWZnPfYGa4|@AgeBKmp7IgeyY))(r0Mz9B{ZPi-67BM{>k=0+;r#s%H@whJJN zJqHI>cz<~K`9EzG>5g>`T+kzmeSj*KKW!1l-ZdzlUn`Pl2nM$g$RtjJ2N>&TMS|5HSenTUx+}Gt zLkYzoJ|4lGqvOzws(4&kp7*z=u|I#fk6JcyL{GjD$a;_<$kP6^=8d-Gi6SxfM9hD< z-KnH%r&Qw3NOEyyskTFHO0-{YBLYFhEEuKRUu`hfHGg@mRamCuX;x%hJJ~B^_{kp8 z)IpU`Of0UqR}y3;U^(ySWpt&aJIMey%MGe1Ot;J2`pF#59PPPV7~ym}CIFy^@86XC zFYEmgCtv9k5D<>KZXA5qu}mZYsw{hT&SH`8x-)JD;`8&WfBTE0gn--4mhE)I3aa)) z!Hn&X-%-?mdFRBMpg+%QNG6}o>Fz2wl2{dw6vd#}FzxNGU^ zW=f8Fpg7@`LEY-na{I}}lB2GGS_XG-6)i&s5YJli6_MUC8UvWLyu1wPMU93uD+i5Q zj31kV!tk5M#|Ds*=7Q|q@riTLDg3_X)?XZOeyrda`OunX2sx#Z78}-R45vO0GnXED&p zYv(6t9Ea{R+;{d(5WVD;ZW&dAJB+UnNL^&rAJiL228LrSDj1U)TBDO;Y+&X1fIWzd zKLu3!`*%HePAjIA-ZVB{+=L?3|Kn_GP!}+SFZu#-(4>Yg)Iv?=>Z|uAO0D?Ub$4pko@WPYYy9XSONqP-xP~ z_9UR}dhFqCZNh-AXw7bU^3lN%J%aW31M_Sbl%KIdt0B>wf}rmf*3^u{5A6Bvs$x`> zlt3W_lCfqM*u5vR3fkN1K0y7lGS8sfIK4K)dYVhuU@?RPXdyKac`ET+ zG~?W~MulNDfmF5nu7LtWTSVzIMaBD>pFVveOBiXe|JjqgMyDYt`d$e{gV(o;(`h~Q zW(oqs{+|VeBP9&#H6<6+K(z$S=F_{SH;Q=r9BlPj!=ox6N6IHP%iH7iGwmYgoPz${ z`9Ers9bybE0GK}oFgY_bGq7eKg%YMi-`O8S{DuSNWy&xO>67^_J>x)(HyA)-BPQTT zrkh?`iuv{Hjp2f2bJ1rqy4Nwftwi4PehV(oXSRQf=^x&Y_B9Q$*@ZK-FjL@wI)Xf9 zJ-d6%tbqUK%^PqppK5F4VTLK~7X?X}>JR;r!D`eI%48s&wiAFn_U|AU7nWT1QJ|3k zs#IK4gR=nh$lc6Mr$PVvrjaPlwV8=wBbkz%oUQ%!Q@kY?E(R`xOk;ed+Srh_=99^t z9L<0sxbQnn&Sa|}vkXFHap_MuE8t4Th4XYrVcDeyy^xvLjIF6USnA0PXJ=*kTZgNO=UPf>ojEilI^ogNy!-0{I;2d8VRmFtxqp;Sz?~}m_E-XYcz1fakQ7O+=(;49v73S8~=mZ1=YhF~Kq<-9uZzB%Y-?LP+ z6`Nuykki&yZAN87#WVQ33~*IJWhK^>HKD*yC*;76z+{?4AcTV+smWjkWQ6KS9xT_x zx}TEr!88Shg}te=lEQ6?Yehjph|JN!YRsYX3`jESiDPeolrClwx6q(D?9FO`V6WEO zR2qp}@8r{Gn-yMP|KiEhr{ukyZ^v`WuR?2exD}%17=!mpHKSn?s?FNLta$Gg2ckNR znNlP@JOnKD`RCKMwPiRnTVJIXk(J9W?klKaJq1SsEES8nCKm z_J+^pUuKCRMXmIQXRPk$hpRt?FK4;83lVT2-_EDD?vQ z4+@1UlW7{tXf3#PoD7nGueXqsR51~x;5TZWvWCmd*K};mpRyYN^R-3<*e6H=AxNTt zrX{i}2Iuik%go1fHXAwW5{9)}QtDj>EA;in@HwI8gqX;a;N|ValtyaNXU@q6=&)si z=#-L@s$o5w4Nj}$CLtyUg90mQIy_!p-U1+N>ag3Xw**Sd87w;hE#t(K0@NS?=tGN` zOA!*&@YTZwoGpg0eP025nnib{kVx0LQe$Hvxm~0k~vyU zdoELJW{=m~+Z%*Tz2jO2CC`O=dcb>yqn8oE`8AY+25wxgOC8X zi|2jp6w~3Dz^ak62P2I8Cu>nM11)jPGde}xjPfQoB8x+H<)1;z-tEh+L{Ct$SV_{_ z2JQBeOK$RO(B?Z$k&c)Q=seqMia`z4tug9d>GXlsdQ`u$m2X89gi9n$?DT`Dlu@cO zvK2%8qcJ!r$Dk`uuUA<8It`>^cGEcw5x(nY3~##(O6!`KOi06XHAbR^;ZAr#kT!+n zP{6Go*?acbBP5YdJ=)b6aRyJr+TFGND7b*WxcJ7@6ah?Q9>Kh@h{#C8C(vT6!d<>B zl1;u86l~cr*MMSaF1Qs>akJPucG_LK{EpCcd&V>MBZ~Q2|46tjcH62oBvJ7HQQTe| z8fKpe!dk7py~#kx1=ElsWer=}@^BKLHR4S1VuNxHXcs_+I^9Z>z-oimtO}b~*)VN` zl%XZkJ%cscT-x_7&$D7?49Y_CDCutoOO}%2c7*L1xq3{XD*`D`1_;prda!jhR?PQR zHHDz58F53Ke^$s4?749Lvtd)Z?Sf9!5>PX7C<4v?c%_P#TjJ#jB zl}6C`?{M{)cv7h)BcP!p1v=j2(I9z$#955DEAZQWS`e_dS$bg#FSZrn_zaQ!==pQ* zLqzIn7Q8nSP|k)^mW4{w#r4z)Im4n4o2Znxfb;5BH;gK-rLE-f8Xtt|W0v(aD=}GA z#rW@tto8yVY_a7c9Q9Yw@aQB znR-uwptNMv=X_M1{VtayZjRcY3M|7396HZ+I!?z$u1*pHVu7lDJGOcoXR`{}llk#V z=i>+kVeNnM-QDH?58wT`?`Jm_jynpt5ts+hSelhA=B4tnIA$nHc>X)?>6VLPY|VpY zK0G)0SdM;qi{`GB-@g`@-#_*A{#BDF1&F>YK;Y(`LGhc9oBY-gdvxRgG*Q4Hi@Lg2 zGG|9h@UdJSIO7Hf&ksMH-P}Fucqjprbi!c163P57%3ucf;7^u>mwHZzwb}e%S1=6u z{L=;N@q&il4Gs?0`Q5m0KE%i4eAclQf%5}NsMT9t;pKj`y4S?dL=(QnMPb2KW6f)|~lE@fG>#Z4Pu-XY_`HUTC_n<4rCL~yZlmauPMwa}BEK^zm>#TU{+X8Zvn?-#9 z9LC4r7zNxB8~MB=zc_Y$Ehjgt`C82mY}}bY7(;;;1AGRuWJnxEXVN2(M&7K_YQ?d2 z^D6kMb^Bq?cy@+~|FbgH{J~oP^U8+HYNC~YpVBb|sOiejG+E{hGK02|kZNBcAAO@J zP)iYU9!7%dFxD}AJag0!J}#M%I150EI&qdSE^e7#h$zf|pm@!VZj6g->o7>U`tZuP zP;G9rrX;Lyar?|&!g6z*leNCH}lEZw3!Zi@33C-z6@Ur zGeP`eWPEEqg0JT?KrDrp7)Q^2#bM2ZCiGsCp9<}>=i9Lx-^9Ve$;+AiqSNlX&$kwp zP2md1a`P-9y#?>gZfs3zGfk5lqCj>IQQu8vdtNS+5)LHXmK1Vymt1ZW$J=WG{BkX@G2-^Ci z1_cs9^zLlN^LKff(h>lEoADw?Jp`(vL15gU@!wjoH{WM8)|8z7mKvEl8;x+eLq`Ft zELU6M!B~2BbInCmOCL>;GXOm=B7H=1%yRPl?wCAS-}a|PZ8r`ruEc2&@*n-S3@!3A z`Vk&)A0Ju;e9%>);RRd~Vedz5u8usO|GA#HG)^4JhombqT18wz%?su(wN+v&^AD|0ES6P`>;JzWQy zet1R2)z)+IsQ!qjn@Mt1hzLJH0lDfU=?Vm_Q`PUc=HM$JAiDna5CWxcFMO_vyVYYg zQ1B(9iu&m(Z(@FNP={Lb>F4jF%6T&g>JUMP6NxOG+)V`SF~>e>)|V-NRWB;Lf}AVfk_w@6fv96lq#17Kl`xT&7kmT zt)NTdo+gs74+9+{D>{k<*08VtWrW4RY3beP^-oI#Q*IUd!9|n!H(N= zxO%!bJ}qzZ1uwShurCO3H$w) z>fp(~2{dsV0YN-nIvL#x$DNv+K*>W8Bb5f-+S;1ii%?$sy{D){By%7Vv}tzA2SfL8GdFu^qFV=6XN-UotJ}?GsTkHHY8QSjJj1%8sFzX zcQi{E3&oaeT}{y(VKt9R?n(f>EbF;kvAXU~5(&yojD-ea36ou>1FXR=4kT}2L}h0z zw$zTNQo5 z3kMB2bzPRzjmsFC?PvDR8c0M73cGLQsD?M#OcX3!rqN(K>bt>n678mke=5TEnorvo z$+h!#-pQoaI4&+N0ZNI76M^kf150wMZL{nIO`^h!w~<=_Va`}@`)DHn8f9_`MMXV_ z-b|sZWX3gp)nVKNrv2h_T;%J(rp$AL7q}AcW{8U$|Ja9|af2lzduduE5X$##0u`y4 zfC&nWhXG(TU#NkVx_vW{S!fH}6#Q5bKjEj+FVJ01PxX{0zqr^;ft_e;;3f7c2^h_J zYjYzWdupab3q11#*rCDN(h5mVi}+*PE_sc=I<$Oq3o2}t7YAQt9H0hD$Le@T$gn^A zOc2&o7r50bP^avX!H3jC_|szl)7-mx?n9j7eZ3YqM!)z!Jg2q5P50ncM{jXpn^Pz& z{4g`ghb^{x$BAPU&}owoS}l+g3aYEe+;ba<)6In#QbfD126RL(bEx;4B5xyPicI#p z_Y_s#sZH4iGx%vd131Si4QYb%HLV{y2s|oYYa2 zc7t8!n9ff#E=^6Thv0d|VAE$heAKp$N-;Gsbf9!tMSVs3X?Z?W1{a!8+!ThfwcOCy zeT|)UzaQZCQx+Y9pd6Al7QA_V%OLNMtoHI50ER$+H8(Ru1eO+V#>S<3e8wY$wL8wW zr{ahG*8+ZOyyFyJ_P6vs=V(~zIveS8#7b<^IS0b;CJ!qjBg0?QrZ5`(X7cQ;BeP+& z!(hrJ9MSIRDc??a`WFyH{tNMvp$;$ft%bu>q!3Bjt}9;)K>_g31l>jTM51P`K62#aHWrz!R4wZvxN=CM}W?U zvQ@n_6L7D^_+q=+JkdXVw>MkEnyVs}%BMWFth9c$$dN}(IJ)xI zzv=Vm49^lq)?E2z%tN;Bb*F^~4s&=aKGcrJzDuj>oX99nO8Y%P-L)8RgpO3>H^E+k zM1345TRZ@VqC#?AU0q;OQW6pX>cxB*nfl7#$=*2g$C|bkrW2R`$22Sk$A516_^!s@Oh__LFI7o~Y*-rX)BsJf? z#ckWMSTY}?u8fTT4@ra|1tH;`Prp@4<`+e#^TIuY3et@~s?-P+-+32L?zph-)jX0f=E>1`hm$XiupgCjl{CrT$h_YtYoK7yRxgIWiJZ zlz;DW-jA4_o!Qv8-?>Y348)*xQ)lIFH(X^VE3V%nKQ-Oq_}tU2m?ox0bainm`szU$ zubv&DZJM$k6gCbhnPGmjpD@OqhbZTI=jF=FU@2gD0<&`9_t4=}?uFVzdYgdf4;Wj) zqv=s0Ccuju+I&=cgmY!VjMc;>pSGyCK!7ZL!TbqWDHuMX7M~E4%W^N%MAPwTWrdRO z2IhegOsz(K(>o@`akO!ACyF|&5gj9sWfl6q zzsnx0z_$go=|mQ@F2F)Q{T`L=fgYKeo}-B3qSN4_7nGEsNOU31`y6*hgomR7rw4Gj z0k;-#fu%5#VTOs)7HvNcJt|1&g0Wp00V)#`5s3l5E&77zz&Q0@DSZVGfdo@(=c+`Z zGx>VJpe0bF#|4{`FR7=8q&sqc$pJvnzU`5WUS|p%V?VyOJAX^cp#rIeb8XiV{}VLP z`@J%{UqoBkak!Pq=%sE^-!FKXt+cafZ5SE)rjIWlsRypiR4TV>hC^fYu8R_rF#!vg z@|iIkzQnwip@G2@Je0q$6@RwQ%*=r433w?(5io0L=2sK>IjiU;fZT_N81}+3cb%6A zG$$+zuyI}^6oeq_Du-6}Z@%hoZy~nORSpw<83OITWvq#$1Jm)q(o!a)z2ga!oE$%f zS{w-lrWXNw_;hSJA8#|nasTX2pcttqdh}1T&^I|Ps%r`6z7iVef0`1%(#$l93@up# zR^akT!Cdxkd@)+xK;9Mi`CQQ^yB7wHCv$L#7 z`7E%txhu3Xyxg8Qs$*MzvoH>ZDVljekL2`8zWm0p&DUGiOsWMHBCeg-kuhI6M=?%u`b;U)hPf#flV!$;; z!q7Onyi?t0_F>f@gg*IRC+z7u8aVEURa)A7Ouc}QGNdTXFvv^KLJ7JFj0s1ZP0iW{ zRHTVy>cs+f^NN6BDn4I|!v0V-9!6Mg=^*Edi@E06Q%Se-T7>bt zbN(;@#~1kDEn5CHnoL;(CLN9P)L}{>B_0p=;4)R)64;j2Pu$dbe=)K5pF0| zxOS*V%}jYF)o3J%#|MzM)7qi5`I zYuxlBVeq648xLl_O7r`2NAso5d}Ot7D1U~Ox%I~__fQvzD<`t=FCxT=3v{n~Ncy;u z@WgZta}ew_3A6GR!Rg6Aec+~bZeS;OXaOY!3~zwRm;U2N(79z&wmX^B$b0+XF}aaI zr~;O>(eZIIyWa>;JRJP|#DLss6y+;Q^_KI;xkP<46aHk%u%#2w+O)LhhvU$9v(iZ_ zT)5KB@fzHlE&Yyc=?Dvuz8&!5zkNVSA`f^wfZJ=!sci@FW2V*>N;3o!f! z8DnZ@R=7bHS;K^`FZ=iSfvO8mCS1#fGT4Uk&v{qLoMd3LBC4r>#1;elRQw+WoxnX+ z;DDP_L`c%zoDoi=V+W#_8KOT4WBt#cMo*XmAFjhA^T+^gn*76mbpz}m z6NkVc4W>~4Il+@7;!%T|FJZ%(CH)_P4`I(c?uRI2o!`FSGNt3r~ z`5~s|Fv{jeLrB8k56P7vAat-gAj4xsxLeC#95yuS=#O0fnPhCi#N@A=;`t|EAH=H^ zEV}X06E%4}H#@?r*bcV&3l!5@KPS(sC-b)AN@a^>Ku|yN>?kPb!JD+^HE`#ST=+`` zqH2yf#02N3UZ?29QnWSCdud>c>bmSaUI%_0Nif{LIqpu_uW83wFbNQQb=CVc%Rx^| zd!S7G&ZgsPBW8DZw+C4DPNow42_tYQc|O_L+V+J!eHHmO&uh?~ovZt{p?tSrajn|X zK8%@oQnLaBuB>MSon|ZlvzZ!TWL>JeZxl8?KBp-gdUV0N+ODg*O*OotRVfvhKHa0s=D}1fl)m6vT+hUnnDCdBN z=UCra7L@pFLOK31((uby?gyk@=hcquL1*vBETtanUIBuUC*;r zg0DND;RO8XH+#KZfYRDmAvVAc-9Zu~GXH#4o&jqyt(FwfF0CZO!z+Q_Pf60kElU<4 zn158%(RzG{xzv_^O@Z?nf-YO=z?N*@J_hpbljEwx?JCf#fpx0IA3J{YPf~&}1elU) z+K>9c7V4k3EY}r6Tmhv44T+^qR=+YYGk&T>tnC1D&3cWD zgJ}uGL!xJGqE8t~xn8skz>*a@9&`i3E*}1xC_30-|4ZUqvn%Gg_h$+WH3?M-_NpqU zf2Q@8Lq`jLB365Mp&@LKxNVC%$atctl1PRQ4=^e)a#O_&A^B4jreE2I;xMgvP?O=} zVEw=I7X7f`$iQaI1TH(S_s{h!M2%S@*MwX7ThzWrCN>+m5v{Kk5y=F)zEBk+gOUZ> zVBaR-uf4`R!^epK@@>{g_S-b^FHhE(2ceu^EN=oH$=(C+5?DQpZp64|Qp~)~l9t@P` z*w|RohXf!B07q1fNr?Rt2#+PlZpyb4c~-%#WV7G;;K5iCg((f5CG+$)Rm9d&e$vj) zzqyF2EnM}Av~!*~iI2QScRBh!{aYBW9adulRW75X5vrYdlCX)Ti0nbq|ChTOpthnJ9^KmKG2XdFb~TYgf#~kgce#b6XXH|= zs}Euqz)MHG%%3~FMyoy0-TZFo$&Qpt8e$xp7NVMv#Lnq4*U zTxt(Hvo}1LTH123@ z@b4BsaOOre!u#a=C*~E0@1ael!3(@syi{$d5(EV{;J{>H-UWSrpbhe&STmHi+%}H` zf9(Tug~7iOpPgL&+r!m+)SU^{wTeo*mW#zPL1%L|9BPMw`#a@7H=S0n*(?o7ah17f z`Qh&Ja+XJ4ap#{RA|Rp2SK|z(90sS{YyjVpdv z3kE{7Qdus~I#g59DGExQWos+}CqLWV!k!%ycZjLYsA_tS*c($)`JQ~RrbcSQ5;IR% zo5(T#Z1Y47wU&A%G_xmUW(Ivvls7k|^hAvhv5i$i9kSKnB{d`6d}^uu>qpG^QdS;uZgO3iuMTf*f+`wT}JQy&8fBlMXsV~b+$Wt9PZDVGP{~%j=>J&CJB4YO;7)Bfg}*ki zNJ;kxmli@#DtLlU_z_to12Hiq|NM~(#H3<{Sg#dIgd*VylZ2Z45moLtQ&i;D9zHE# z{`FKed+yJ9?ZjI4YR+rXys(;u-W-Br^YUZ8>GMjvtoY~o9icO2Tx(yGbaF(0#M{+4 zyYG41G#J!~Dgn9W2`aFDV=3MbY&t*cKVWYAQDsNpOe!X6mmsw zUAkR4T!>|E77G^AtR;R&v$Z4&Q9vp$4HrTyO5i18Z)aew#hxbxTB-q^Gx-w+LILiv zo8Q9I-o)Z+0U=%7H2nz((viUC5vmc2NhNfZ6!KKq^W$u#i;Udc!sPvi1EhmrLpE5a z)t`B;M&I&w+&JL38leYLLkm04R8g;^vAm+w7G`)UYr9{?bl>m03I_P&x8}rtUNPwL z=w3qZ&Xm!nXlCE}`!-Oj0#?-mN8SEdvxDQ?^XTMXhK*wxps*LVf6u}hpE?!>OvpUNdFy+NEsM3-E6 zJRmJIm8i%9&K2~hvAG19dhd<<^TFcb%|(Ee1A;&yP~LH?F(Ks_O$JLu;3==n*Kqybz~dHG;0&?T66ArG2|!c!>-)%YfQsnkLRY zdG$ZN0K{roSTi;OT{Sqk5W_;&+cI2bWTxjqCod$%w#%FlugQ)?uMJUrU(j5g{vA4M zN*_jEGg|SrMJU++*v^aff>G+p7G>?1Pg}99zujxAvaKqKOOC3Jj23Q$9R&r-HfLF$ ziQl}BR$_|0wX)<;kS2pKlDO0vWR|CfKhMkT4S9-VVb6}Dj78^IJ^i|hor(*2r{eTN zNtv^LXCzPL=+Z-7uis!?{@-c8Rx8Ft? z{4xbTH%j%2*!N3=gDQlLYMOxfD>IDpr{~sNxxRZ#XlhL?&mPVAv{CIZ^DR$I?tK%A zeuV?J7xrf-;kA5~jS9!EpiA)#Rc6j5=(oZ@oJATjGjnUpSKhTD6~^E&DQ@~Jq4u6U z^KHb8R9`d6p@pvWFPjO^xy2_eSSr#fVs4dxw*6A!9yod+CUyah1oT!5vCPcGg8f$r znC36Q=>%A>MS>eG?%L$+*3jO}=e4cj3pxu@HNu6KG<%%E{Xk2u|<2Gi&Ey1E{ArEs5vX+>V;esH|2 z<||LpEJ{hg7srSqJH{7{>+;OTYrJ|({p5TN6Nf1{{%^Cu4#h)#SGlnX9lY>w2@roMc%}w>6-j%`749G`Y zNEoq|&F31*y{34_03RfihkrDvXatkI)Pdfa%WbS$+LEkkh!Z37B|oWGIqdYg55dWz z0t0YNG&~aL$rGA1MM+@UjkjhdWWAzDFShR8`VhPNa3`4~mK|0EF>^SPTK&nfor#6? zwt-u>!MfKV4C75)A%_xhSMu?NffE4sOP{c-Mcirnox_TxD|3)LPbFkwJQCFd?;zRB z&=MSy8CwI$vJxN4&@|$!Y&LO%4;#M2R<9$Gdg%jKGP#!Day&T}?t9x8+y{MVeynh;87ANO7PnT1QXOF;!0uX4nMu6M&_ zvMJ|+p0v8Og#|qsIeFX=06XApD>yL+>J`vYHRGA}RjY}lq=atok5$+DP*5rq*`aiq zU%qK2KCBG83VY@m5@)azjJky`jpL^oTYl+_9olU1z2#7dvN_dAH8aZBud0!4GRC0fJ-R>H5k z6r#^MmQy?B&qJyu;M9RM1+H(Nl1$}Mu%8GF;B=w%xFIYSWGdcsD4RodtZbk}U-Qc{ zILoho?rdDm)j}*A&9-zKY+}-CO!I@!i9gPI)z}D+=VlM;c6$>$b_oTE`H(p+yIs3Y zR@OQUjOSjkU>XCSQB|?2^_Pn(0gPn8$$Oey75e}D#}TV_vTW5l@au+P@v9K8j!UYi z%2@p!(h?6ay-s#g!r0?Tl)Dy-exTYZHt({AvyTAPYWMf=*KgmJIsg5@3G2H}@P~A3 zBxacqDx>}4x%N+*cX}$k7NHgoO}AI|UuxBfW7f~uEI|(N{2y$cbxhlRxb|@?LkA3J z8%}X|cXxMp*WnHuGThzW-F3LTGo0bhkoULuIp@EVv}r<`CM{pN<+?t%rl@bUWKOMU=bh^P?%m}AaWCJ8b426xP-T1C;E=$+i=uq>_2!3{?tbp*)m>cCD%8*d z1AqQNO=hGH0~Zk)43vOy5Gwmt2Q(_*qoHv(7M6@FiqS6YW$RM!m7_(##xK}&r~z-o zpanYNLx5W`J8PsuBSnf4su8D`2XsFG%g^23e zw#g(eHkcr3W|7Kk3d{%?fLx(?=h%USDT7aq{?F;l!V(Nr%YY&zAm~abN9H4dEAjub z8|jMtmisC8I#9CR7?C=oTLk_5HeZF>qE(PTG_lkI4RhdwNTocTooTnP=Fpi)DF@-H zvE3INPom`+6p>TN%MjTL*F8}tTX*DiNJK~yF9v?kQ=!}tsUM?tQ1zlm&x*(&Hp0b; zN`tD_u%t+YQiV!?X>lk2Nw=4JTwQ{@8l|b1AUEB+5e)j({nYVSnJ%9)BOsUu+H^xb3?aNqPv#9bW+$7h<&A=02QfU4d6W$j==e#!dlMr$9Lq zkbTjXr~zif|4gAjbm-Jc@=WBooW+KV`3m!^=^{?!sbcA!vc31l}tWr$KUEQ-;~Io~lGL8P;C+QJ?b<=E8Aagsv&_)Fp+1*7R8HBjSXv4s<>p!r9Q}=`5Dlt3 zCC0X+ z{f(z@bU_*U{|=(rWF?Q|!jEy1C{Mq%$`vq_IbgMZa8^xAxnhimgn)^@{UL;bg0gEd z1aWbv1E`*Z=S%|O?bmN4lig59b)3yoH7~M82ER01arwOATYFG*t^8mGpSpWmyT~d& z*y#@ZuNBi6NID_>-yczl9=3>kj!J4iy$cjMXnitGu}?xu7T`cB;;S6*a~TfB9M{rF zsRRErN&zB%plwchL?;L3cf=atqK@Nol0qKfcqv4VmBWux(#GA}pog7Ukhr{~R&kbv z7SDu7FZZR%8Xw8D@mD$a-!y7;Y3>JFPa7Kl(?I_i?KaI~(fA7WHb<^-f|_D13nb|6 zvq}|=k`uZrcm#H+58M!RM$z+)<;mM+RlT!^)s*R%3)M~AcT1iHN~ANHO(QkAH4+_e&y$X{b5~g+dmS&cA5QtgJQF# z=iT=~=jF@5i29BFG9Hs$=uYulllKD`yuY;IhxM=@k4QqP;v@!`YLQZd$Kq2+Ef4g8 zDFlZlE-YMfL>g`Q^`I^0cX^Om1wHN{lcWxJ(QNm#; zD@Y9JI$8{ee)r6KfI7y)Gd)_b-6ie=k>?B;3vRc;k8`H-*0BP;xiDH;#>OWCT;+45 zdi!?H7#o~M28w&JN1t{Ys7qi}1rsF0*WmXs@?{4ZW3e$k{L;W^HmY9efriKT_hP&V zEQF21HfW~P$bxUOg+ED$&;=9VJY`#GYR*G*1gM-q_`W&sM&{$FTb9RF1aWnHp1KM= z!29e*IPzRe@P~$C=U75X=#_xWV@+Xk=*=9ib|HK(gY>+wzO(O~x5sdHMRPnGhYRz= zBn)PiH@Hgr&Xv}QF15Or?Bb4xT?qI*qd(rgMptV~Q(KS4-jeyBgZI{MyvifsQ$)%lgKJKW zMF8M3P~su}qa4|{5W%w#cdGF~@gDC!y25kOP|Zjr3s2w9IAL=hP9urxcpCK)f&kj# zleA>TB8jk5{&}by#L{Cx>|j~Jpb{Kaos!p&J6{8L5L!W=)kqperS<*-D?|$f8jrs> zdgykX^^(K!wZ`c7#OH1lG1SF3=-Mhna*UpS)Tx~r@nQy^jNqax1{{SP+GTIDue$2C zYe~4mQ{z4FZ_wLJSB3DFG;uE+gnceI3Gd?2IoH2akhVe&Zq?Bj?Hs&cwQ?=BG#A&| zyH5MxZ%}>5$yz$Tr}8x)-SlywMm|L+!7!(047WNT0_cOE5~Q)Z;5E&ORc0VW3e`@o z03Qlp0vF$ZDlgC?)%P=d*W>Hl*qPFSN*b2RJ=pq;LXT&&xu}q16V+K2O!xj$t~5Hi z+}B3F+bx%0s&8b&$G?&X>tWuPUu_9X=6a92eIV>0YT!eRB$MrEUF(ISJ<)m=2++2X zLXz#YcEy%qAld_aqC;Pvgy4C#fzny~RGHtrk= z8`mMDy1gDpdJU%M!ky3B(Z(gP`vtA*eyd@Ak4eNzNn^!JT2ru2#_A68S{914RAdD3 zf42OvsygqrfPw&xdX>R{5Dqa*0gOy*^~zrnz>VlAcdTXbFPaij3?`Lq@woZ|@Mz4< z&3gfP-XE~cmlUqyR9R{6YDC=OUDka*<|!r=oV5hKV+Ud~>72o2=1%f?LO0|&rl=&LNnj~!$Ind12isInjn+gV z*I{ySrIR8yVD^S=Pv9@tAgcP}G-j)qynJRpEJ1Nxzt=yU3px=zWFAtKm$etSAHpIt z^guvEeG2qIN9`gp>Wp(cYXCHKA|GH=oSkvqQrCN{u+kyS{0H*vU@T{@h;QD>> z4)l0?X{tN=j*cjScxSt`ILinC9h|Y0t5+X5Xa6MttN=@GlYjFQ-tXC5{xbgh^9Nkr zd5;H3Nq^T46B(Z_R(fn2MNGGZSaPD|DO?_F=-#eTx~?mN$2%t3%E07s6`nk8zaajM za(*Ur8+a18(yMprg74^~PgEUc-A!#LD!v zFabTn_imjC=yN`*ydzOPyKJQUqiWVvBb0;r&jqmFnO{ypqwOh+`p02Ep=;iX4GciH zI0gcbZ#+AleG61)#E70|=OJi}Qup-lPD$@Ke$Vm=W^@M9xUk-?BsCVtJj-%Xw}Lkl zSzG&Uug^*e_+%q}JgytN=KOR0?DE_Vk&hQ$QpI|~chXxy8whD!wpg|+Cvs%5Q`GZ? zD%7=R6L^4m43d^q)0aXA3td8Y0~}iGK_z`8R*q&bgAq6B-*YuF|x6V)oSde znrH14ChJ8O;1UrX^aMaNo^Z);5*Jojfa+%~9M0D{0m|PB;3E9?s?e#Ye@#An=jFm; zdv`06^cWyN*c79Dz;F|}8QKObgBgG+#V*R&xHUUK@b~TZ?~nJ>O2pD+hKHYKAyxgw zT>T%&RPfmTcvw?Csg};>x(JWyHzV{@wS!Byq4!IQq%-r8G{J4 z9jGr4T(M{Gs)yXM@yPiuN|WhhY(>uav(;2V7@y`8Lnw&^C}qP} z{;n1p`N-^f;dvQEjC`3o)iaYHXjb1BHuM(Sy9*Zbx{3`xp*yiys`Gpoky7JbD(HO} z)z8zkW?bZoz3X?1urg%RecbBS+zaWZVp#f^Y5KM8CPg|qD7GNC+HVqGru(lxop@%4 zw3N)lnDq`F*@e@uNN5=9LgwA00cXk44EJzH0D{en=L(^V1(&YA2Akyb$Gm3@S_9^q zwScZbPZSl{(P>5MbXVDyZHRBBOAj|IQz$#^yb)x0JY8=ruOXaZR=Ys0QeG!ovIt2j zit^+d(fEs7?~o_0c;3qFU!FlGh>$nqyuH=0$rxL$+tUqKn(#j7TotI)bFrfEgB4?s zo?{pQ1J2DeiDpz@Utiy>4G>HNYhsqK%j?d{&hKa8o-T@}qd!&f_SG=4eCD>^Nw{b; zgZ=AMHnYzj0_?0Z<weqz9!zix#Y;Okl$fm1vk;Iu6^s=_-07N;ouKcK!E-sESuf> zFZHf#thb|>H_9d6q&Rm@lR-%GN9=7S@GqX{uX6kV0!@vktuc8=ng17b3b{X0OnsUF zhD4ef@AZikO-w9?qV?>1r}ORWgrpdnN7BTfq^NZt4{rDSKfB|(sM|M6-_aod^u}yr zmQD9gd`(|P`rD`4v(*nDgO;E|qZ&uN-LxBI8gc}!2Fal_mQ9xbm}JV4vXp6{f<%8Y z-iPO{<<6VARW_wuwkW4AP&$wBIcF8Vd?<$lq0V3vc`)R`?p0e#E((D@Ud6J#hu)9= z(gH3si`6hLY}k76g3)f{x4AXHv?3xE6Q}0ksXqVk`Gz$>OHwuQRY)Y3LwGxsZuf9* zTJl>^&4VcI8pboLnFkSZW+&A+Z`9>z-L=2(|M;42-r4q>gXjI_#(Nu@xW`rZA3O(0 zu?_V+c3>d@krb!hdP^^0lj^;_Wl1u<;_klTCgAtY0B8Wlo6XW{qy`6xgP{?#v@+c5 zrNJnI7^PdiVbdoqv@dUDKRric!Js*qWyviZP*>UV9eGUG3taWBswLLT;b$JnyBChc zo28Y`dw*9hKA)a5R;*n85%uBnD8C;#eh}K-Dg|vm?KGFmX8Eh z|JJ@JZLl7Fl7On0w9*{2H*~|qzL?Gx29SM}PJnYR%i2%B+m~0pT5lrr;Vka2e=13n zEq&VB60J?K&<4#>%hkq$MNYH(=l3YvOv@1!N+8L-$iEyx0Iz_0-#}GHs zqLmlQhVDCT=Inlc(<59&C@>DN4xkXwLTZUHiBZ7`V{iq{m8qaLWp>+13&12)NL<9YU_J9*C-IHc!F1Ce zmQqR@;77(PWa;IoSj@VZZkK;{kcQRTCjuVNbh`WZN5Z{{1cF3xS|8sxc`EK>lLWl5 zXE7ykz1?A1K-+j(!KHY&JKT@Kf^shYAX;fcux#7VW#v3WLs3NJM7>P`JpcWf8W(z` zN;xSEcVeG(yr8$2Mxnmywrb^7mAhbXv+wXFWGiv3+{aIMoNd!z)M^}Mk5TYxmad`A z@U&ws9L-Ym2^n{i;K4j@VW@f+I}=lP&L_@261c7Q2w4P!Ypp0=V=c2qRTrI=C>VWD z+YJWqYNT~B6#Ed<$dfS;;Iei>NmLm ztPwn>)ALC1_S7^rZA@RlS?@M*G#wvHeq*GPiZ|Zlf*H|6CE@nw+_BQEWdv||!ND#P?KzGu!NatQPO)j~b$v z9=O~@Jgq`5thKYvG;F>U{vYOhHBYIq*W-CKPc!^&Qv7EI+Hnw9`YVZEd*et;46P2E6|R=(ub;( z9fqpS(`y58Qvb~e==_F5b8%nUX%GTzri4FE;q?+}d22neMQ#fdAUWs-mW$WhmADcO z44bL`I4Im{Ta2x|_#{zFBPer2qR(D|d{SZ;Hs6qWJd#z{E(91^-oWqj8eN5!h-rEO zX7~xng^GezdrDj!-vNt~0Tr~Ax|Rex#7@zNN+q7^dxZf9&?zxXtWSgG&ZM{1WhYe~ zWc`%ziwHC)hQC`%x9n-8w{|RKMRJ$%l!AiB987lST)bcF`Zerf2wFbD$nS zxe!7TGjRWQN6$#!mJB|uJsBsvf8*ER|JHpu#`vLW&cx&p%WFwK09tRF%+ofCjV`c_ zvhgB$IUmL+ozmm>?Nu@Z9*@sW0{Y!6%Uu~=Nr39j$!K6%j6U7UPc}GI0_21@&bDW zuN`kIc#e&tsPuyjD2+}!-ivghin~h@7C(zSFIrG_Vy)Uj630dA@XB0|^GhoS4`>S1 z!|f7o_0gw{Q<=0P-a!CH+E!$JPB}(~d21xCNG*}iI!zz%&4@yy*QgLw!x~VaXOc-4 zRlESnx}*0qRGwfg4ubWVWASKVoZN0{YqebI_!fDr5#{EM-~cJ95US|J6yJ`hQhIkj zsEoqAU*5{xvr}8xj85{x$c>MYP}^c$WjP3jSgm~Uq#maxT{i6_TA%S*59?6q)b2;T z-h`YGX@PE|-=7^E9iTbbl9Zjp8<6PAiMs?%U9brv4&%+=iZA(j$A%UlYc$8#hvlhk&HS z#z$7SGol~lR_i88ZI+=?S}&8qQ%UPl4k9x_f;a>#y#1DmFgrSn(n=W<99cYd_42~T zb=f2Q^6x~I;f{9s3Y?q=qNiZ=Kg9M@@{y5dfd)+MsXAxH{o;}VVEmd zNN%V!B)}ffrW!-mOW@+6OU#f`qH!}ZerreCZKZi4#TD`G+eErJy;`3tq`f)GEpb?d zM2|N&y$Oy=hLq^DWPBOm*pG6SD9@*87%HrB zDBS-RZyK&^$@GZXOleoPv$9iPdsz^Iuhl-Aw?H{v5k%tz-k1?|Lf&$ubis2ne9+I% zIhp+07ulZRIiAo#_+zHBzaiB=w@^Oofu)&dt4$uPk<^$vhR2#)>orkql8nzF+=`k) zqmpF%y;>2=TB*8minnrE)ta(?|(q23dAGzcP88@{ROOIk3EH!-3A&1#Y<3OoX&aJoI??RHd; z)LF#}sC!Lge$~(u!?z)}V#7147ykSUcmb|vwXGw((HeWqlBwvUx=m$4IgQKK&OVol_K{$Xo4; zYXoiYuK`xJCHkRk^*EDpcnV4LukKqX%)fskMSuAmg_0!nH|tw}TW}y=S2|a$L>^Bn zDKvB@R~LbE3gBjbmW%%v-;Y72G}~?L0{#Vr;VUW+*6^h$+foC zA0QM#b5c!URyg~Gw6%&gZ>(WUSQOpdE|kY6N)sODkuuk2&F(wAylv4A-vuUC<^~SA z_=&ZtX>OJcINb$YsW0AKto>Hlz)^N!U4h>oN-051fHhoJV?)R0Di!G}e*$Xu!cTaZ zQd-v=K1FZI3R$bW(g7_sUh2TFVDK<_Mnkjf_y4qeNu|I}qf|RylCf0ABqysgvg8t= zPG~YG!g!veLyRw#{DkA1Cbn8MWT#>zx4zZgfqej&P=U6f^z4;fEEZx4zm=4mnOP_h z{412Ia5!v3Y#m<|n=OOtZGFflOt-aDJbRc%3@I)^a#T>UTheUbcSCrjRY=|H1rt4W z{^Zm@a36bdpLi_lT0Hn9Ef~#hV>MOq;rJK&F-qM;Hepe?5T>cn%<(X|MH|WI%i^5+ zeTYoO4?#xRG`$HrtjWaw(gYDfAFc+&EkHZhCDINq|4f|ZQE>^53hB^ADvN6JO%;>R#d^9PuQ7)Y+QxnR}YW-CAM85HvLhuS9l zd~YTRf|Qpi{oBw?VZ&oMS3_{3hiW%=u=VovE~xK={gcQUO^pOX1L#R^O29KT=Ow1I*S#Z*GPw6<|5)D~4HUl>I~)nHLjk znH6_0RO&i(-+e&NA<~=ZO90@VLO#pH!66pFd&LR$Ua0h$Qn~B_}fZWXuplR3+v;Y4yhnkpV4=XKu?<5>?mgy zdGe-mRO9$)H(AvBEJ%tySABsRQ_|g?O>Jfg$~fVFnx1x4Q$HsOUTQ*Ecw)fej-T9^ z|Ms(X!JKk=A_p}J&grt3yVLVwwVsFZ1ZhXUtz^vlKZh7#Xe})tfO23moA+Gc;qy#${~PyX@t4CV@I(sHQ0h< zmY((@5)GG>dJyShV4Q~Qo(}uywrH{PCiYmzgD?4;;8^~{Lp0Ofj(AB-)Lw^kN%x?y zV_Z~7`Ep8!p^rc*uBG1V9Q_^i{+P`%cZ_iebT;MPG5Ktp;>x0XnygRqs@H96-*hdN zY$%fZ^42>XY&&|boF#Nij$kiB$_{r)I48=jPqu0ZRq*AiTHy(dDa~f9q)&@*d}pWl z%B(Z`8q%GHZI6oO2^<+uf!-jb$I&Y1qU=^5hLh#jdXZrwXQ7SJAKg#UM>)8#lOfZhbbIW$a}OW~B>OACW@Hru6%Kr0%;&!w)@my1Y|?okQe(DG2Nw z^9C`(;j7N61WNv^ORW)smBsusjW{-9ORF$Xh&hqHBT>yCg=s9GCM%@_E!^`;E)g~QfszfrFon+{?fQJbEM;#gYN_5Ds%PV|z*;A;yXd;=E6bvm!fe;_4m zmrLNgyErkN`Q@RFWd;Z*S)VI#1)2oaVD+*`W#KNMT#4wrduyEW5l$t;0?8nF=KO^c zeaTn!IY+Im^JEpv12FpPj-!QOy4A7PYy%PqFAnx|m|Y?iYY8QdEO&2h)+2vFTL#)D z9}YcF#CJUXc)G(jm!hn-1z9+p%u`u4;zfxCfpF*mnykE^pI zIQ+^+yE~`PGmG_N#SVZSF0EQV>wz6Tn32T5_oR+-72;k?NTz_x-oVPO21(}d)jA~z-h6uw15i|Lc7i=IYA}=%&o_T>e_lJf!e=7^o zCZAzcVcwZVTKy+pjH-U|RGJ`+z!D1FG~D*>fksa6=C$G?hx;1C>8Zoo_UTkEc{gns zdCV%mNl*(4n9rdfVQJZ#u#oy7&`AWIlrSt`2qNVf^Vnb3)i4_9Umj96rR{~|{G{_- zo^|6KpX9aT{Eb}_ktIt6C3Y<>DxZ{?I*-OoW|^|e=|=D!=+yy}mVN=VXo}@B^Rz0N z7#VvDrIRfhwN%O%*Z^$Sy%`-hFto|c!-GQ}a%-l}2$DG_IOnj94tRgx4bT0>mVBYo z$T&XNhZV#G8_L?I9M(voBN~!9+VS8&y3o3DC7;jg;086loU0$L+Em^Tr201U>LL;E zFR?qvn|kY*iOn0oLYKM}mx5GmLory19DRRLTQf}`7l(`71jtY?77g|f#v^k+u;hR& zc?FqAG$p8hyE%r+E~?tB|9Y{M`0KeG;amcNLQ8Z&ON6GrS|xvC@Wthq<$wZ>RuG$W z0B?t=<~)=E3@-Q`5pqDZMu79K6+$J|w3=r$hkb!AekaL{5#Rgi`lpTR2}bqWQ%1l% z{DpG7HvCo`a z(u>4^slq@SNPpm_%qq$WZBR2#U4-l$)zgP-pA7VyFd>f+5nI@0%K;LxCJ;-_yJdQ zUG@UgSS0K?Sn3`)#{iMG0$nnv$ECp%=I`7P&w~fQ?4fZPHp^mQatVw~;|EvnFw3?i zc#ryL$W9qg5edr7bb~wfEu-_5ZsIwOf}wr_W9X$dmf|hPwX=)24g+-ht19yA31jZ! z6)@c{T9y5ez##jG)I-{ZL*4;fx5#)EjUITtjS-55(YV4G%X^0m>iQDwDy(%`qnTed zPVA1UnHdEX%Q>^}va~9PgPYwgpERx~6+ z<9R##s#1K=rtgQVn&Cqi@Ih9+464ji0E3_H$(15} z>ronjljXGC1DWM@LItQAH{@>as{Jzmbd^yv*co-`gBhxN52o8aKIW# zvo&DWOg8h$WV)~E;frSKX>ek7LAttQp)vt~{`u#SQg~gs;gFivWong$A{&x?Wh(vF z${#-o+G4J^h>JQvvFh@ZxPwKY(dTj(JLMIliC{I@v_FR*h_;c~^uC=pElWALYW?^v zPei}6EJvmoMr2%&wd{B+*4Bu7a+RD2Go|eowLH?3lExm7GKIJQ4vrVmZl^P+-Eq3^uK zAbz_HLl<%Y7A)d-Lb2>>~YF#zqW&}xhX{tsX-9zBa* z?8XnOvZ}ZY-==3Y)#8ScST@AZx~??>s7))R+N_1&oCM!^xgbR*On;kq^*3hw1P`r3 zmA8($O->K9V8dWXZ!|q~Ni*A=Ak6v?ke0Xh^F=GE`D?fJ0=x`qF-(|mxduDznC%MF9gS69!D=UgVc+ z2LDanQ;e>OM9g9CDc#I4o+xY=u^Nr(6K0!Fx4aBN+|}zW6KAfSY9Rxn77%)CZYf&k zR8{pU3GO@cv4%$1M83J1nMR`(7LWm&xVX%xn0b3=$fmP3#sK8WuZzY?X+V$ykSNiF zE}8jnm=QidE)Vk!iSs|tkR`WkYu5VXLZv0doi0@avpZk`R>c4ya9dkS!t*EWDOD3p z5wAF7)SvZNfJv~4QKTXj4H_z{P>+>1jdAD_4-XgDLoiaC+{5IMnSNjaUHE@Its?s*J z&7MElk4(v17O#+rTDT{spAduji!}5YD`1WX_}R4+9AV#zM|w{EjlKI=qWYpF@rDY0 zVhGbyQC{m+PUd&^S9!e5%*ch7(gqsNb+hrLrElE*9^7q~YXX1`c{RuXItL8+V;1}S zRs!r<@<;k7fETJ#t_pC01ZqDu5+r?KprvV?6o#_c{Wfw?ixTg02s>8M3*Lk0(#?M3 z{XlNCpRCZvJKTk%pZjRf2GhHo>Glzi{nGox;YZ#`MkC~LW~s7Hg}foY=(^Bv;rGuI zEm-?Qw*%Q)A)G`z5)AgGZolZPOtM)VE$Wi6_k5&Y5Q!MQDFMR%*8} z$}Lw$7ELk|oxhW9f7*>GWu+e*vM1 zr$FAw=~`J>T7_>TC!okeuH>^`uk{S1Q9krf2J>w1pJhcI#!*+>?u*_@9Rslx$c$QqW~1n`B`BwYGK_iI9WCHYiuFUk$V#y|GMVv7gKJ`p6H-x)-h#V}B_rTO z@dP>Jwg!M7KU({2OEFInHEMgAUN~iqg@j-i<>Ju(lR=r!7KW==iE@!x1edm6>yL%` zRgEl930@zZI6Eq_-dUC!Nw({}@Qp?5zN&)m27E&a9O-!5=rW*PWJ^R_IDUKguO0)y zl7Xo>AQrq2UDt`#mjISPbe89CUfCPa1BywFJaj$q!ok6P!o@YTvx`o}&CUfb_}}zz zyY$OI1tV^3qq0H+1Q>R&vmQDq7PQ?;v5S6lgexIhY6kx`#KXn0{6q>D$L=5~Q|y${ z)keSdG|e>5oSo)cxh8O~hlwYy#W8GF`=GMAnfpeN6CX=MXJ#w{tS6hDFK!SfCj`bv zuxX94{0Xu;h6&{#E#L}|r)zTQZYF66H79i49OHm+nsb&f{w|Ymz|mP{G8M5?s>>?6Z*-O zAnY*PFAEjWRsexb{S@C=KPvw?h`s}&2Ov=o0~4kG#WpJd5r zIN-1lo#cI3cRsriLg0IUni{?a#yvak{rX9k=nQCipblcZFfILs96g9X6-Vp zlYxZmy#Huflid8tfEvo>9~&|TfjA&vz$re?#qKd4?q`SD zXkbkTa^K&w)Z?(66%x1zJ#@#dA1yCG@f#2}qffuR8vfBuckr=^$R(DUb}mkOl1Y*# zwVRc)@NzA-4FLC#FMQ<8NAa|ax|A(N>Zzjn9IZ%KAH;mC08$cP5)Xw$&3lj_!Up!v*Y$8n-7r2m_ZrAgCULeI zA4)XBgA&7%YmrTHC~$XCuxCqzO)6H_do9IULkkCrqu!pl<48Mubu$dUF0Gv_X1b;s zQVOPHiwY=o>E*L5h!hN)$|p6ra1mP9aLKZYH1i1BYq#~Q#fe%%si50tSd!cBW+fxS z+UrYvM>tci^YIzu4r^7q?+eC4NztqWD&|Q{*HShVy@<;AXfDKO;`YF3AWc+{oYPw`yleS)S!k0h#J zGB2R-{Ef`C;${%()i~6Dd0ZErZgarIJD=#ahzlXYtM$7sl-oxBlSr_0Y-Pe0<0w{> zV^UlkKhpABJ%VCJ;+oAv!YZv#hFiB& z{a-CWzSO1HUFkEImPY7{ZyI$y&!qJ&Q*lD;jg0U#c7Ut(0`H>l1yNu8h~3=xQc~%p zy;iA>Sr$nP$+94rQoV%L2Nr*pC_c=}*@8vlDS+67d+U+a4EFcj>eGRY6CKe?YAlc~ zQUx(r`V5FjFVm8h6Vu<5otmDLXW6XwA<_&7mt8r5w6;o=J?bI5JNzYwWEjb5? za|R02)Rg^(T+q#ME6$R!bv5BO*BCo%QZ1RSAw-(Jh2!$F0Rnaete46GQ+}suWDv---jmrk85`$ zRvySinSD;C&AlmcWHHTP7@|6AaeQJs$$gHIS(Pp6-M@->itRLvwmYJuoU-zKjQQng zfn8nyB<9Xf&IKa?na2lo(*@SGS-Wq3z?e}iAMI)|C+LGSwLlAjC~XTlSJ-d@(*!`Y z;PCreoK6@U;@N){l#~2q-`nLq@afTxTDxTwWwPOSf0>#oLRM5plBvfQ>-xdxNw}XL zq%%<&!W(L&@-OXUD3i^-7O6cvFsT2%4ofox^AuDFLzGcW5;1Xw~^{^M4K<19* z{cs^DLYCYjOTy9(Yb9{Xj_af!`=V^$HIm7`$uL{n<`Ds}Aag0f9#Pu5m-$V``1?z= z{ZE17%;t8O4GB&&fwwWL$7zGVk$AP@7fq2Y%e|6B+th9~MHwXKCoYiB({}wC+FigBk=o}>=%VUQtw}cm=G-rbS%C!B}w?^#rcwZ zreNMONBQ65^+XFtZ`HC>!m5A~+itJhdB!9BBhPH!LyRUo!)1^urw!nXOPO%|?LL-x zLr4`XsXHgr>e#?bg&&|U$)s&%44~>hKqLpg(!TO|amF=?#VggN$5$jh)P!J=2#N?F zrb75Vu02r7HRmN-mG+dLFcN}dzu1YUVTc)u5yu_hq+O+9Ldvf^$-D+=epQ=$5D_hM zLvLhA4vL1&jw2)>0Hky7ffW`#<6jLuItIb#wD)%b7})a35~^|nWz*7ZsQ76xZl0O0 zbyfQxERz%XZHB&I$Emu?3SF3AD7u;{gWYQXQVLk=I--PnICU(ebez{WutyKxznpW26^vfUe<(lnoeq4P=A++q);C zI+wCaB0piFHCz(4hkmADTduGO+~#`h!$d2nwhqC$`0!kXHd^_L7>C}s6Q2Bw&e7K5 z;Z<0gc94y&t`YCjQLU8$d{Q7n(>sRS+S>dFH+IRmg5z51`agT5`v&DS8()-{?2C>o z8K&4KVIko&-%UDMYl&R?~qd zN~FGa8oRI(2Dn5`&O$`N~?9Q+TeyR>GlVfNZwNRL`=ar4* zncUqL8AE(-pNnT%Y;2o1j$72#_m1O<>(3ZN66NKqVmoUzA)e5)~H6 z8o#4KT^F27s5=bi1z)0>2ko=&&?lnvTB0bNrn!HzT+by5yawG(4K~8)%@F)FIGT|3 z>SoWLe5Y^zL?oIdno6O&elqR=!NiJ$IeC02(;hE^lr+B@PN1D+p`9|;h;!gWzw&oB zsSv_U-mhC;n#$2cxqX(jry);LX2W3BpfO#7|CvOsF}$8~g(u&h$z=~qGpw~cIP10M zd6_^ZfbU&iB>+PMMZw=laTiPH#MJU=;7&i*M%V5)L4L2Q$ubRB1LB`c^A;H<9ao#7-(QQCzVnL8_fi zn;CV1q8z4S%GO?(XjH?(UEdk?xl6ZX^Ve?(P!l?*0zndyknra~;Jy z_x#S;XYak%UMmktD2vU!EiG_O?2vJjgW9%?92rsoRgITf=iz^X?|>2Q2ykY7 z>lJ36JBob3#3LNu_!ehK;(vBc9!eEsnu_JZn`rlyzOrH!!d@%wFBHoQuL> ziymF0P-EKc_s0B+VSB0wKcL|-UviZC^Xs;iv``?wtAkP5p>?&a!ks*wg_Wqz5xxl; zK`ikgM^dGPY`IoCWyb-hv9<&*9TjBZ%W8>E7fSV2({p|;R@ zW$;`KkY#c@1^@XCtyWV(E~!#3iN+#`$tYZ)R9(D3IY#%VeYU`yI{C-v)vNENy~{3W=NQGcGj3FN$GQrG*x5M~y>ZQgo1-o3UWV+T<)y$1 zBz`AbXstpZ6Q2?XaphHSW2svtkWX+X7K!rTbMsVPJoR_k>G}WIpRyF9u(HiY-TgXp zd3238#t^sY`N(3g2a@ya_S-60iV4wW%d5+0=%l9IcYFu-ayeX~Eg~Tjp%V?gEZb6N zAjtdjLI{qqTXdXLot>Ql@N*P=3Z*&uHEmwHY0;*rC{D=<_^(yzbqoPm0}R27!38MO zH>V^CJS;FcRm|QNLhFYWYKFEIF4_7$`@sG5bTL6mN1`nEndW%WW)sa= z=XcTU2~WOr$fDq+@Zu0mXv;VBy^C+3bWh@8Q=wwsx1vOwS9MjY1GcAp9P8PPX5ciOcg{`wr!FZfzJ&<{A0Bd9(j!bkOqubD@!pvWy)6 zE-CploqV#|kqKGJ6>{1#OZzR3PEn0YwUqDiNlNqxXPxLK zqVLQI9r43LLPCBYCL|G5Ay$}%nlJcqY+(@>9lm@&P!|H5oJHjj z!~f}vWGlGj4j%-`kl8n!4kfCabs0=yO72352OM_KMfFg}i!_xolUCP$Lr)al%gjHq zLE{?yi|_@M%V8YFHbeNyJvsbW0Svn8nq&34nTmN(qNNM8#rg5-yT6u9rG8AupM9jq zsW^>=I*2nRo-AhJ@l08}ZynPlD4;cF6HrLCuRVC6-TI&~J+;$YWHOl6^_(pF^d}3~ zfD2nIM{+11nY8u$=X=*08N9uH;Dat6 zku{oh1%<2gF-)_|go;YDkDN3d7Q?Qjg6p6_j3gzV;DW`J^Hdxg&5%lNqak5MahUkk z;uyB$7{iNIsK5tZr z!};+s0bzA4-)Kb$S*MQ()%#JU^24G3xCfU#3Mb_x$;8q;a;f2@NOQZodVqHgB!~A6 zZL%p7JA422;XI%VfV>|7#X#v70w}Ds(oO7yf4chL^n)HwRB8}R?WNG`8rj%HfqzeR z;5cOZn%-1FZkhY(he^`6=qVgiO_QS35zl!;${l11>l3E)TZ{g0wpx~k5?ToZI z|7+Sr21b##iXH8picP}a_7qL8(t|EKMY*b$WevS*T49|BG~@_hH?z!|#q9wF%iG=}X`n{g>&>XnP7;ejlb{m~q)u5?KbZyKG$gg)n8}{*2M_9anC+&$3mc607MjrSC!~IHk(? z@X9oD6gDpcSxffGeD9#vDOu3e+ ziUgvEyyyyT;hVAlnwPzWmyTsMPj#bbXv|PeuEF=b`g`AD`*|5A8pfYLE?Ydu3!(2u z=SD8myuc5A_=5XWMf5dA(0ykcoza7f6GP(16W2M2fbxm9FGEG&{W$$T107930U8rzvL@P|8Ie~OY-;6 zf?ct9<Stdmh?jW~1`K>7U2fi=W3zw>MXga1oMEKUSB z{NtlE=$Y0U_iMDf(E%H8KVC#w9XrNP7(xaxNZ&xjisP%Ew0K*BH=y(7#{M-K&^peX z`EQ-yrbFLe_&gr5Mz!_P^RBvl=MnN}Lp0TlTwFr|@7}+;fGy21cE4%Qj%9N8rhPHz z@VR4r)1Bq?xpS!cX8D;$o2kS5q?pFCtKW!)yaW z4ImIn96sQ!#7kQ0KzAmLo-9#+p855GS4kM+X1lUXuT=7M?^lSliWX}{jnRKBWfbJh z#$o5KU))RPumNST`18lbS$#I29YVEfgwMr-VO-Z`as+S=C#@Lv?~^S*60xK_c9h4D zwG^3z=Op}Lk?fXq;A>C|lDCAKkqFHs>kXDShBE$bC19jU_oj&NcD#hC&AsobhsVP^ z4$6Z&OHC{xxX3=zHxf*<-(j=I_ILI>Ak;9>^`QP53B1NG`~9E#f#!AwtMy#Nbtf=a z1mbnyfuSx$w&y<_pgmUzx7TqsCGB(9OC$#O3jlIynVEkdt`C9EFVmOPWojBi#sm{L zw`8!10ay>feE>t7nNOLV%+alARkl+{1^+SLB_DUCe2>-49lpQf1Wi9txn_z9zXH(- z>%&}0JelBD-dBXjuKC3oS=>VrLFzfXqodFXVQ_BA6gH!NAwYEjNe;;)S$ep3DbY8+ zqDsEbY?H9*Di75)mlr#2CXzzIhE0>3Hz1#@YX5866hb1Eb)q@0M&*AR&HM>INrBhV z9riP|xKj*KeEzMOdFmqtk!4U`6&x?3vkd(+*(#D>?$G?ps14JvQ!O`8BP!s59oK4AifUw4d|hlF!c1;}^}^|?M!WwTj`rIF2_92j|Y8;X1{^A&iW z1BoTU9|{{0RN0KtsaW;go~6~*wuXV8-<|#v!tcV|=RK!dQoeSs9+{>kQVtVK4wti= zQY`78@U&7hoOSTOqU6x~=wHQ`v?69^<=-wrxmkFYVwv9U#zG|zGV}PST-UqT^n{t^ z36Obh+Y3m&hKHt)WQj=f=picveTZQ&L2uOGc48eUP_gDiUM&1SjsfuKH;MI zTqIcE6~DUhkD6rxz3sQs14wZsn>MCIA|fJAOiw42`zI(}Q2Ad_`P|g!>>%jhK>BZSzk#BI^8zTb*u`i5_Z)(xj^Hrn&vx+0& zts^*kZYL{?dNBpxOf@~S=Uwlpij@Qlpy|VBA2I%FMHN@1v-P437K%?1&}9_y$7HiK zeO3?U;qm)Oz!s@{#BkJ+luwU^-o`Ae`VAiUnpO)EuVcdhKpn=&$hSV#&XMN;q2|rN_^z2Fn=ZZv2JZ-qHHD zNQXtSpBX>lU|1=$=dHVeS<{KYn-M^=ubuUaa@sc6CGZr+=g>Y*Pm;{t{?R$K2ub_z zG!0LT>+Oj`8Ed|0f`AZYnZjpf*3Z)jo%PXnBj(npd>Yx%{vVK@TSnw z^}O6Q`Udp^=G1u=^(sIy!AkuY0JGoTRUk?Jwafo{zvAhyQ?~_KukzXSL<|keZ0>Ip zeTY<_K3{I;@J9`;9-%QL9vklm-ge(frax4%g?|cFA$jO@*R0ifJ**+UaBHeydp~MW z8Mc%>nMIub@WfnSs+d}2s1HYu-7J2{b;@Y!yc!#;y*c4MQk!@zRk24?z$s9%XY{d$ znv8e+iYUmQ#2kCIn$}l}!(-fgYH4U)Sm4tbpgUb5-GL~n^zM-~&YTikU?ldC(~ZLO zt16z(z(QXzYPev$5o~Zb4&JGhsol&>NuBIGM>lTIAv_5Vq22jmm)>scuABQu3n93n z9p)XUw759bSGH_%5S6N5AFA&`3LydBj{{}*7C?QH=YM92;eYW^zx7}rhQ)^xdF{aVg_yB4H9w><$Z_LUul86r z%X^u<5A~u z=Ez>LA{?iK!laAyw;NT&ig?cBZ<*?N2OB6kI@=!*k2kWHt>cn}o+h;UEBB^`h5^9d z?v<=k?HkD5vjPGj?bdz{0;@N`#9sWc76P9Cml#@D#t>5`&rK-1{7Juo2MEPydeaIb zwM5shK6Q^S1yeQsN@T5sns&U?xdK1PwsH^4=SY|qG{8No`~8k2*Fex0jq;yi(n(Sp zr_&%-2RA_*pQx+Ruk}@>W?EH)vDl$2DBs(E+;fG5isG>sXHUW945J$}IPoqric1XL21MZcD$6Ba&x7gDuc$P`2lt&&qJX?$8Y zOmCL)iu>W(vN7dt3z;spGYQGQv-z#x8Axeal34r#{D&Qhq1@?svf-^a5F61-KNF} zC8?JUQkgc`09P@$gl#mQT$C zYzvNrtj+e-iqk2PRWt6t)K?4_Od5S1kW;+)`?AdQYB_il7FvIjtPMIq_vWuhIfj~8 z>n}IW-?g!-Tr|(ylm5o_X{GjAHFjhv_}GyJ{E!q-;f}IQC-Dl-R+wtK8yc4IqO}b! zOr~QnFw@-CJQUi^a5#$G6w1yZUuk&8FD$s7g6d$mddbXcOAwdl94qzTIG`Q#DX%XN zD`N;FoQzGZp05?+Qhm0LqhZlKZ`m!rOzf&)4n{7J3yYrZYoJSgxMvHWIY}D%no1>t zUC$jMwOXY1k33d|8xAtbDIw-*8~q8Wm*5v}L4#J^ajdI!oN<5j`442@@SLZ?oWH~R zFBiTwGO?KUhW}9e9aD31PPz0+7P#q4{n<1FmEU#}~2h|8{M6+{7am zx#WJMMe0qP_cHXz|}EN@>*@QSV(q` z!i0~-e5r8PtOkQT#Zo+U4I6e(^k zb$0h8Z`hl3l%B-Hx& z&V1KQoR73W-X(i#r%>&8sPv{y9~x*}>te&^8!F^F6mJ9Nx;W zGH6Ilr}~va*_`5FC)r!x5c$KFlt8}_H3la`X0^;Yb23YkyazFnAmn`4l!G3N{MF#` zF>4q+^aq=)RA{#!ZQLOoGMn4W1vL>@?UDvg@Y zGyLlUF_!6l6664|#q0CXE-mEIyH78ceg0)n0C zoEMfiqWj%|INycwM~fG^H(?lIEZ(qETb3*qb0eECTJM|^H_9!simGUyw1@tUm!16d zx=<)xJ(#H8Ve;x}Xyn4@{Secoo)t6_x2Qn8OXo1=W!8@yIPN5uMJIW$gYGRyErUjp zK^j6sI9H*v64Lmj$yK%eUTj<_i2UaKzn6HlUW7Q!7bpJEdzpIYXj~8C18;*Zufllo zE<(s%9*N3G=;e4DZSGO?*Flz3Zx`ROC@J`0@RY#N=28Hh&U~M%Zs%DD!x=DFQ~sQE zcc9)i1iGW(j=zdw{ky^~3YyADnk)vmq|YTrP($Rx&0x~3NYNp?gZ45iv%OX&p~8Y8 zpJ!EL((`ujiae7aA*1=%QY9xdIp!8Fhxplc!2vN2R6^$&o9Gd|-Xv&-C%dE-!(63hZ&jhtSvj>w8(znjz{Rr6c5>^)NTgFX`Fr2==0Ow zMz4MquEF>k7Wa5Y`RZMl% zgCaK}5rB&w`26@!iTWfmuuc+SouqkzYP3l?OJy34kC8cATFpykl}$p@sG?R9fgi*m zLuzn^v4o&Z^vzpuZ~OnQt)z{l%Lq8?@q`Yp>^%Feb330NW?%B{y6TZ|aiu>!`N&YB z_x1Jp0FAwwxjER%90VMao`OT2y3hd$RiD)aDlr6LN<1pnA)qZQ%5Y}V=k~kD#g;}v zdUiF)Ra~cT4P}5yv(jM$AvF^k*4>hF&8G_jllTNO@s6SLQT2ZLI`ZNw1Y*G?D$FMQ zJ4gG+2DKkV&_&$9K?r}YkSe9B8GRP6hvaGgi|bA_)&@tku>+@HD~ye3uJY%^`-h%D z(IUO2^CyG7Xhhvz##Qa2s;nl@$lxY-`{bgdmQL?C5|eY;;A_Y(sI?4b9^|3LKvom*EI55PfYgKmf-xhyGF)uiR;X3)*e6%3M)mp5N+ zb0r|`zZVK;d=?P*H;9)D`}@bq^?Ilzr(;FqDyubaLqD;i*l2voJDb>jrLjDO-}a9A z?#Z&>)mt(!&px+pAcg=_1*-k468@#jXu3KA(iUQ(gN?xgvJm?5C3}*)znlpZ9Badl zrk0kz6k!QNPs2OR%10NB;`6(j4cU{){SLA-7uh?TX@c^mU(?s66pHS)T-K9Z8<(Df zp>SEhNA&u9II#a*A?y9;?&`{lfJIBp&!1CKLCgH*3-@Y~UiI#xdCm?HvwGW|Kc6m! z|L*SA169o;nY13zr%h!xjnrmtSUz6yy`|zR#2;9SoNcjh;s( z5icSr2-&x@6Amj*+reEvdR-r!!YKnu@sA3HjM6~8thy;EIgA=w?&rx8^6wrMV$IDx zEQ|kcCrYqPyI;R5H)zI1x-JUgS;9sp5GOaDSodn~bra>~yS5v#7nEbloz2QoD2QzE z)54frNK-imwdsUuHxc~gg@C;Xft5;^v(#JLB_mk&c+__Ans8~7fN}NevmNBafz~Wa zsmfj_KQg|K<+7psPHlmn#w53pu+y7}8A~{|omd19DXlrEK{IB^r%*{)8UAcvz)}9(8sGWH zkf4RW--pI&h`_n5gR#o}@+_z&Cw}fgFZ_bjHvdC&Z_~rOCwR&aE{HEEg&DRc?06nc z_{;2ntGFWExW^9_{J+8qnl)@9j>pu*6V9g`99Vt+{CS2eadL9f3dvMNBzPV9I-VRj zB2=HfJf2?OcD-f^2EpDp>WeFAXbb?<5Qp6ke`jaMip9>zh!hhO^SFtDj;;_G;~eZJ zCStq|9GPw(eh7v`f2EZXM2RIIG|u8IpB;)L6SuIlGXpx9B4T1nUjL%B1aNuUJ$2?w zE9RrToarqf%m4nYWN)!Uk7_{kUpn`Yp+|XkIBsh?L?WnV#B5Lz+N7DdwS!aMA|%IG z+hM}55)-x2kxpFVk={A{_*eQKNnBS>USyZS@T4C#>mb@C$*sYGSCNh*dNTC2;M8i; z_gnAQO>Q2!G=2LWcOweog-iW+?((VCr+k{yrkwVx7lpkk>Vd!q12l?-G;7+;((~G2 zJOVVAC9Xub_N=8PEh{T44lsxX_{zayVd!{x<&h|R*T?p&s;a8A+D%af1>|*gb#Tb1 zq>!fgsLfx@Mm9pQ^@x`!e8-~uF4T-0 z*Ou5!w;lr3yzrANN?d3V?P1a%OZoT@inx|!X##%?-eDY05wzY*|EJ2a@>^|`*_7X( zrXX>XcqDqQzRZzl#`9|hH1WL<)$T5D)4tN53Oa>Z=dkGYDs%qoM{9MG#ir}{x{eB$ zlQY=cI9w0rra-($Di3mmEIDz|vfZ1j%*fZ(ENX2{0}jk?YI^{I85kJI{qZ9Je5V}X zv@<`F$yqNQF<{lmZ2Sk&uKk#;NInONW5_QlDJd>BRYFq}=luLwbW>SL=|6fqbT@ix zY7%Pd(&0agm~f~?IS%5fV5LF4K_?^}Tc|Z=`h+-87qx099#7t*H7zw!Q0d{tmE`!} z6pLnV-#>?62O=o0vDQI8<=evt|MA}$8@7?Tss|#^C|%E(iPP!1AKy)PHSg46;cu;Ox7PiYrs3tH2c7R0m)$BqhQTu@WXPWz z)d^_n7zjZJ5*sk5F{a5}9qcTq*9x(Nai-9ZcyC(?j0Z6F#}MsK%k!J|1j7qvG|vbn zo7#0fyMgem2^zy~PeAqf?8Fo@+OW_c{Uod{CqhH`L;ZXkl2Jw(^gJ^*jFaYAo<1cb2ix$YjVnJ`^z?19D zCdYMzD|9cK=tCuX9F7h{C5bKmDe%sZtD(9)FUHLeZFD`0-B*-YtFE@CDSRDqZ+|Xn z33bnfi(Jy^bgd(bh(9|s6MtxU_#UL9i@eC;y7&YLCcR`6=K@zMQ`V)y_rh;j$^4#X!WBWP4qRPbfV zC4pQ&G2Ag!AY$c|M-{y$PJc%Wd|2%dltQj+6Gj6WxuX4KZHvxLSTz2Z)Qc?cS+{qs z?ljA8lzZaK`r;jNRcGQXt3+^>Sjkkwi_<=}`p+)OMo?9xl7L)qslVXnXV|?lf#(z6Yw#ujiX#L>Ir_6U8Pc&+W5e(`o%q zrqd3OiP;4r)es};tl!AQqamVw#ugSdZjP7d?bbVeAC1^DCih+TuDvcUR!RePIN(qz z*L3To6vhV=sP<>eQ_`QxUAwF*rRYRD2(!$_l$mrYI8Icnex-OR{zk!(Tu4%PdI-gn zXune0+TOlDxE$}Wn&SFCzoYx;F5YP~&-X43;X2d+)l57nY$=P#Ud_Zjq8{?D=z2A2 z0pr_*{?_N;n@h83)rUMK%}vnwm(jD(emYY8Xwk-w`2^*ns#xJ_v7+qjjk_(lyYBkN z=eO@SjFG05i`Q-)>30jCzSRzla4vSo6I$xgL_xpJme#oJE5Uxm{T)rf&4~L0r2KYTrH=DQBa>&&$6 zqpA{q=&K^QCmnvuS)eL0X$28Ltx~IRJBEL(!E!qAVC@a3rlzHJX_rejeL0WuKkIsZ zX0UjEPDLdJmXfI_fnHM5(w261YDpYVB+?FWHN$~lRZ(gN>#SsPE$TnZ*4F-B^B`+G ztTvyxJGbI!h4S@iNuaFka>QjO{zfVyp5L_z8~psZxgi*{Y_+cYs)iD76l<-$#0Ldm zd?w~*M0J9fMXIPzB_)W16t4`utlPH)!;_F{-CnE>2C(QGsiv_F=QdT-`WX^8@v{yi zt{ON~vvvOLq6MnoxHt}bJ+ya7s|$3SVB@HNuI*0$YGeJTvku5@d3kx^5fLP8Y{>%y z5;2}D*#>~%10-ng{wB!Zjr+g6MF6O!uKIz&K@t*@0LXW+ExHVFQ0jV~6oJHVTO*lP z(@C!1y$UP@wu+kCn=HATAg`jLA_gX=qMn|dj0_^EX~@{v=8CmVt|Bopz*LLq_%`0NMagSlBX?a-SV67Ii@W_c7zgwe|E?7bu6qC!UwUZ=>E{g(+%RJY8M5X?ZjWzzBp&& zpBpw)-_3r&Y(cAL_yOerjWFA;i47N#^}AdHQb4(sOsGlt`27cc=NvhnQcS`#(={)o z6UDi2;V#x#EiWnFKV`SFkC&RT6q#c z7>#Q+R$#9|3d`(vq;cfN&C1HEqN=K>q!id^^j7YJ*kWXCygaD!vE{`JR>z+gf>l&h z#pm*4VUDjDh}zS(D;!%@KjTS^`_V8lcN}e&t}tM#2sw*)^1^?Z%eSq2(MXo7{jirM zRa@yj*%&l9K;-na?s{kCM7UbND}%xrTcg{Kbgw`wBuckXId<3}Pw+CPJGWh_k^5asOj6cAJ4?pvz_o+a4;hKmTP{YfXR=l%YCz76>zgyBV3OJsRyM7TCm=h1V6?^Uiz! z3Mr!R-j@{(uTuCyNX`2=idJTJBO-amzWVAPdU|@m#iO3&KrX3NIYFo0bT;y39HfEZ zJ8lCsf8%19f#3eZF~{cu9tQ`fprD{dcYR{wE4d^P2?Ig+4v1%itNoahaHmx?oY4_w zv_vsK)^yQWu8=A`ucS>K-1b4V!z?T;3c9+oii)Vf9R1UGF;}GRnNnrA!0v%Kvgc^0 z=?t0NX%cv-(YmOl8_x_cMe@%SAq}3FpBzp*_Et429^@>fhXTK%;7PBJ`-NU#j{o?{ zW6(2WM=+u-ARfHZQ3>c@R}%mD7QE<`s)hBh1FDNLaG9RBLG0$NH zQNkl5Nk4y<@@2NRvf6s9xJpXkI`m>DEec9XBwV*YPu66kfNf=|t}m;pM>tw6Z)|HJ zP^W=H0WX*g&z>gHvh{)9?@Z7u3TL~PK-i=}Jf(uWgdH&-m5D4Ua({ZC$r>x}i`j&d z;^rmcaaCtgpum*W%5{FU?1|1sOg|r#W7h`(4UC(H!pkEI00a zXX!weU^v|K;|k6An-hmir1NnGtPdb_zUj)f{7bh+ffmFhJ|o9Kq5TQEHFhaoz&zzR z*)jxfYHDhs%imue>G*vZs1#DwHEIkX^TlJ#rV1nx33)g+EZF2NEG#z8HhbSfoacqf zl^3sCxoTGbf3&nB@#J~=`Ks(y)=7}gPR#0E7a!_qm30*afS)?X?mwj_b7UoDbYX1T zjjf&G5J-X3jjF42xj$a>g=XN-Fg2FTP%d z>-j_Zk+S4&)!j75l(EjTgX7_ncJ4mLr=OZ>zg(C+Nac}F%yz?=+yi)RxoKX*w0rlJ z)BmtKs-qdgWs8-Mnr**{1pr+C%>u@5H79_lr5&Bb<7!g;9<=X8#R59ST@jXMuitg;vjH429%+euN7Qx@mZCQ5*v4yn(3!onKwSeBn`v%9Y~k3~ zYwj;?OI*M|H4EpElX`nto~ExC)QAI~Ge zg^SFl2QI;VeW;waxJBA_n|Zay%cG56^^B{(N~r|D4oIX^%5*#Z zg>#aOIK;}ny{^$acX2bZn>yyw=MLv8u%limS_|`=LSr6FH1^oyyYg%`XynlLhr*?% z;$M1?A0TY@`9N&C<3FO>zJK&J(~$D&HExew_;Hn;;688^gpUsYq7PkJntJkA5UELZ z|9EwoPfyK?%+NJHyG6O9Rfs=<@Eo21uNZHjZUmUe3|0F-TfgxxS!0+!oly#HeDuCK zBH`uLOp&kKT0&M&9hG@+q&kP@;x%IZC41P2q_D%YRnLaPG=4l8v) z*L=*3I_?h!r81--iiC`e$y|jds3Q?HlS@lW4S|1@iMe?__#Z$uQL9x6dNbc6(5YY> zv9gHaK=?$%!I5-wVuqljqf=2=S5{Q~1v=^=G&c$u8yl~-#exo@!G>F(Fo3QE(Cc zl6&c1&y@3Dts>l}5kciFqMFkzuhr*5mb4eS+lEE-_k)*9A(3{2p6@2?E4%8?VEAgo zsJ!M&I*ZLe(EZOW$~!yA?AtWkL5CiRnZCszMJ1V18>7p*N~!^l0VO>}p%{Zx>TWzXmQ4{_c1HcRWLCW%7A;NO&x z5|Pw5&ae*%rh+urI|%NpuGld=7jS{ZX9Hw0HCD5q^}LT+i)~k6u(GOMl8Kmwf)^qG z`|GmmI$tWP4SIqt*K^vOK&b?1MIZxJIwQahmC59Ydy4@M8{N4uXpkV`K|@26kZ@>; ztWYcGLiXDF4S(d;u1p*!w2>z#9y222Wn$)}NOMp0dO#ByO=J!#q?45JI{L3y7{3gN zoi+rNPFut7JTEipg|bpw6=2r>;hI8Zg_oemRk}6X?9z9Qt$KY$<4$D#1(Dkv(S&AV zxVaD()$cX$Wy3V6!Dj#KW{7CQw;uc3YNQ-=7Sa_dT%|t(6rA4y+)!S0(SQlkZD+_5 zpE|nz7t*deV_p;e zKP~{Etkt_6YrWaL_`O`_xc?IL4rR9c%6-xjbf_Jo(jXxz`J<&p<5@>Z$!V-&od_K) zyd*xiTNH(Kvcs&ZIwWHU>!*IR`dar>JrIKLkL~^682fwcUHot1{o?N+A|4xUiF_(- zPyivXS4l9HWV}>v7&6gYE4ZZOzuE72VBb4Ce+fe(=)u4lG;Lnly7fTht1!nAK$#F1 zjpH$f-ACq=5Lxd$ZbIiKzTWvOgn*FG#G0g~%z@8^$o4uyC10RP(MyQnV+Lu3k1bvx zm%IK|pLuP;B>(^rCK14gZzL#$pxM#9F&Vn>uJqDPGkfjTY*bRGV? zbn~k)5ntQw`T9~L%}6*prr*h<1{&jOetq~ZhjX&7)}NIrLA{sB;~5X&08q<-C{6R$ ziv_BZ_`F_;fcU0?DaM)d9}Nvzt*h5g5Hg>^KrG2|6ArlX`lAWny8Tp~oN4asz9}Gu zebhm|2EIauZfGgGZfVOqULDy?#(x;iMm~XVMhLm&+w}&<6M&#w^%+1Z(TIq~f!I?T zqahUdO!zW9b&NhY>_#*R6$J$WfbFqmeExXrdo5U@GRe+ZSO+WIqF;OagjGn0a8 zxONU{WE&Jz&|pcnbT92cVG@Xr)}Vp^uP%?dAn~8lpKSyf4v9JBej!`p-8c8V&d2&D3>NIA>in^jAN&v0>281Sd&;pK#jNAfE zzPA)>B!@pof}eDb8_Jy&T2oUK%+w$YqU!aSC>hQEa?9{-~FnY9VFeWk?JCwmG+`bfm^-5eQ0D&QA6VbzYQP z0%+@cXKhlXxp5m_l%4KGx5g5fCJ5EaV?0)0u?D|rn@67_lxTaXd{O7BD-=vfSo=q- z6q?=P49v;~%pZ&L?Oiu3soi#gMR0LgL2;D7L=Y&`ZFntsI743kJx0LY@-MAmn`oMW zOoZoV!T9G{4}y+kA0}wH1F5oZFa@E};$Wyi6fL}m*3xkvN;`514X#*OS<|(jo`6?? zteO1A!fkDB1#)KzfZYw7Fn0`O+TKEQGyUR!ZSMrO@Bw9}a{|`;35LG0_gleZeq^ZWjN{zKhj?7977`Fx=&P zD6pL;^WU?tx}d+kAV8h}hSY<+;}R6QN~)@T;P+;ICP2IPen+YjbQb`Y?*cGbYds4M zjZ|{8`m!a5zGH31GEG~1adFQa+LoFniTyC+MLtg=eH;1)q)|Kaq~eb zs&0_dM5z%MQ{cBEzev#+G|o@qg*tr);rr=h={pWCUrgWP6@eD(Aw^-{(Dhe;xI$Bd z>ANpU-6Z;=MKF+@WmGrAC%-v7VR-d?&{j(m3|z@Ph_?p@T|)AXs$JimLTR084^?k{ zlH;oLQ&bcuHF=g)c5GH+Q&MM&l-OtTB8Q&e(NIzOz18=v6;!ewH;d*$N&-t+PU9e& z4*0)Z5`osr+htVU^}+?}f@9FhZ?4AuS}=b&I6QoK*+0F#S?*~F2d{EE;i7ui4?>UUdkaW z$6S>jC&s^K%b}`-@XYd%6Dl zlCu7M;{Fpf#|gJ_xuD#KcmWe}2hezhs^>2!gptpI4BRjO+^h2Nr1sF0M5(lSN8TSs-N|I=5?>|*`l^G@ju_37MIp3eI!SN3D^@1 z3eVunnrvP_xz4-YS%P6dQ4$mgkN(4Bj>Jpo1*hx7gPOs1KeK&}2bDWNoyZou&Hftr zd$HLY@hX<@9&WRey=IoZG1*c%Y;WM$MLfAOYoN&DNPVQ=Yg-s)AMR}G*&c7pz5iEU zEqx_BQ!Sac7;qqFDij|n``TOLiY= zEr9&-HU@!#f$=uisd~b!;ZVHdypyC}<{tw<%G-5+{u1BN4M6aELKWs_$dh;=00WEf z2|d079vIkyo_jySKJokU05tN3dLR40h9pglq*!plsiNP4lMDja#S|D+zlyk7zyuP;}iJ{(EW37lB!ESZMc)uV%xP*hCR zb>n0mCM34~EfIr^bo2aJ%YRMY6e$JG9bRck9k@}_LmHVo-M)#^LC9H)oJ=uJ`mr7A zw;EO=IU?R1H#BlXT+Ioba^CmpwbO;-A@?cjn7)b(0_iX)Mp7@-tk?01?J}L1Yj}KpTz@qKsLLDuN+z3!vpeAZdoXec4f4TdyY|uJ2iif#IR_R5hSq0HRG<;0FMZ3sNY|f4Gz~O$O8? zDJiKMtxC{b1_;vIKs-p^C;1BQbx3d#`9ybMvXGp`V1?DIDbGXkR>bBaBSqxjlZc2Z zaogcHx~~LG6eHi?-@hGRyRE?8yu&`WxwFG=4v zLAN-K*ZT&5Itgz(57s{LR7|*_{{XK56P$ zJhj(6PmTW#MJ@BzuEcD@(f?6Ss(0Y(A$jxjjVYqngu)DNn?F*5iC6qu8FDK1OUk+W zCnk#Y@5)8dOXT&FQ_{$eP^vIHPDSIiCT!ZE#S{$fz5tkyZwe2?hrrZsN z?|kWOy5MVufVa~fWa4c9-0rXcFF&yo91`OsL8x3hYL7VDp?8;?KPI)nmVpc*lsTrIQtZos5 zhHMk6L-E_nyy<*)-!JD*#kj2S2|D3C*-&I%RW{>o`_)XKf1XO8?cj*ghLa$EecDvU z32TYtioZ-)#A+T#om@qX7LPWUCHC5@Z0VdU}Ix*)9?G3l@$Qg zNEGZs(hW- zUdXGlRX5S%(~;HO3PVi>#~q3FKqd8jL5@nd^O<@ zj_!=vb4`4Fl0RsA_$xD*PM8&o_0{AED|T08?$hQ+(kX1qs89|+&ic8@f`YX27c7dd zM>|i9{dTnm@8PK$7(DXy^!(PmAa$U|ES;R*2)(>25c!lriDz7!3?2Emn=I*nnyRNgB6O=KW-?)=sJsz*y=7PoFmC0J>R8Kr#D44U}Jo7@8b%<-&IYL(1A7$oTk%&v5sIjGwey3HU!cssOn>av4^zn_{3D1U3Fx5JT ziaG#?WaW%+_6fC=x$1KoK}Q5_lb5O!_kw86oaEZ@6-tOQ*kw`{~Z|=R5d7N;A#A_P&$kNjD>G05i zbo<_K`lHj$f(K4Gy|!9mY6g$>^{nw&cf1iE_wabiqIOFGKVA9Na#2Z%!(+2nX=zn{ zKjB0X9~2i&o~iCreXE>zN>kcV{4UQ%$t@ydL|hAioYpYf7hgP zC{r?-VCkHo7g7Bvv2cV%W#Q=~oS-KzdP`_dIUH7Zz7!udOz%zasJd(U+v@;<5kV^m z$SS~;(h|jRHQ^at@c``v^Rtt~n`UQVZ(s#B8nzFjfQCAMEDRO2A+~hhfoEN{Cz=UH zACtk=Gdb_ycYZOem>MaCua1Q$(Ed8;yMBy-1X!Bw}J;_ zb_0w;c(-paHx|$*Cvs9-C`3j!`3}MWmn}32-Hi9JCPOo}q_Ajyo>B!Bn-o&=nro8! zzM7XDRatA3)9l!abcuJuyx`+TXCEu}ZaqESwY9Qonc}?Q&*pu_UA|ec)cZHsaW9-f z^cb_f{l-RD@A16iPV-2KXH<}AuyQDoU0S5Z^(k1&2$w z^m!*wLoI{Z%MCFxdJqqhfoQrg>~Na%b2Ur{+ZN1Xy8kH-!{oO^-f%SdG5P*eI@#=4 z}WqUPTQb?zokem1D?@IfKy;(w zi285E<7-FBl-jJqN^;TaJDdLfeSNbDajZ7KUK1M<5@)~jr=#|mQrWugI+5YHHuIhD z_gJv~mwvV?R4xUs-TUGzl`TQ94Ib52X%%kE)hoZq61K$kz}Q9Hc@vQHgzr*Kx8 zo7B_?>j56k0K0U{EiZz?-gLCwYDpU5r49X13W|z3fVn2`zdQa(c7X-Eb8cf|sFAzI zodpX$RsbdtyK;X~;Y7z_6mE8ccA23UZW8}b zV-qe0$ybjPV}q0P-)#Zn2314DA`Iwz)#3xB{*XZO0{}bV@bv+@;s4_K{;Nqdy1sJ3 zl#ed`W(CNd+!=SE;|X8b-@gi_Hwo=sw&}D7y{6&=Yq6Q>rL~cBFE3}re0y$k78jvi zBF!XzoET0N^~+zZ(gYL6i6#t~CS;?)4;p&TireayNI@y&x0IgKl`0oMsqEz?Bp1yn zLnXt1Lk#3piD~q!+uIDn^@4(hEkxI^h2H*jr|#LT77q41=VpeijK)RpQKO4gWIdSI z6qm|1b*B_kdGzpwCgi=g*phd-F4kSZ%jZ6TE{qVhoS5@$kjBlptY35OfxROmZRf}y zifd?4!l|(xt#J(w3c~N`=m0h;r>Lmwx0}rJ+4gq{&gE9J5&QcDFzo=8d-BT4)?LYW zYESpZ--XW`;p-)8p&LK|ySF}?t#G6SO)%h@`rzsrZm$kl_VV(((8EBZ(TxsYG(ZpH z>RMsj3?1ukju%M_3(g>^xbfqM14n))CBC|BlqJ>?Beg}XgU=^wV8jr@gh{@adY^@# zUx-jWkzC)XL0CM&3i}<~^XiJk&Rlf7u(NqhnJp6v4kkIH7ykFlCiyRBoHW7%Pt9lC5xZ=5~Bpkx&A1mP)cAIsa(4QO)rCj*H_pn=Ps) zo6^bT#hcf4u$oz>)WaXNFRME5zogAqlF#mcB#63WrGr_?AcTn6R zG|pGorMQ%o(S@KD<)G=@R0s3Lt`v~(#lCsd;j>vMXK0ufPl$^oE?gEY$qe>|b{~&` zKoV?{mn5`V`k8?Qq(sXEo`9#Kn8cLfzLV4N*+C-DTiTHn@;OzwyNsG%!U5H@=9eCB zlT=VBX~C)Jr@F!GSgr+>=5CN;0V1K&>JMO<=@!1X>Mn|FN9Rle2aap9e!7()~xQ zwBf8E=$iozk;v3{;g70pr&IzZ4RILYLqOSLUZz{MoE0Jbehj|SL^ps_4JWUIi{WXF z)A6&Yv(wFy)m4<$s2Tcm^|Rlkq@mA?ztX;QG5db!&TDuiEX#qa-&OY&5D@m!TooA=V#l0P|gQ;yFUUM3Pu2KR@peKM^QZUM6)3uCi0@Ex++F$Qb1t0Q0h%;Z z6JPllA1>P%Q1a%~l8EOtihnWtzWjGNJDJb51eP!!nCV{=UzZgWLSw8#LQ!;BDwyj| zJ`)v}t}IhFNT?h0iomw-UcvF8w|8X9{QjMTb9#x0ZsPzXG)>4zv{JM00b`gu1| zZx9m^d7k`j{R$PzfE5Hpq(1~f6PMTTXeW~>D=8tN6f~I8yLY4r30Y;iwE%%%FGr_3}Dc*w6-n>IV%Uc0b0Uux$LQ#s$gQ(A-EvIA|rQht_+idq`TT< zv!wcv4W0G*b0TPMU@DU$@RG5yv4!VXxf!zLtgnmtpLyALcNb=2Q%-#8Zrv=s&u2Hr;es49q{jttoQWH=O<=S8hpqlEXKbXU>UhMKYuIC@R&rzn?PqEtE40f z$Uvrau79c2*?J*G|+Xh8Cj@+20i%a=UOi43)k|7o(?A4Ye51y@qaZgM-*(elc zcGjl$zgDb?L&gp=x&Y=FIQ`IgMe0m&HCh6q^*uXWLi7okQ*SOBVnpNx+^upPrZGUf zuf%bxzO7tQO6nYx{8ThF>Lw;Cz%Of9(KJ8Jr`C+IO39++IB?i^;I5w~sy@`SF^HJ^ z*r2TR1}DQ?*FGk3 z2M2EW5Nwc_qeYKG$wUvtD=no686~+#neMEvt$kfD@6@fZCWRjtoBDpJHVxuv(9QI` z`%QUJ8HWMgWER*q&Az`Z2p~Fz31wUI3hXiR1zlOgYw7sZZ0K3wYm*hP6Odf#VF}K` z-ZroibjGu@VpoD-!p~>wXy2r-6}-?z>y{NN#2(z(aLmx_@FO1Y(bL=iQU#({i?~?% zaK$=mDE_Q+6KeCl?B(?pTzItE1jNkh-}6$Xl5uRmJm*Mv`7PUGl)umtg2SW<1eu`0 z|JuI=LOn?B5+G!Rd|hPHAevvm0**v}xa&x4snzEND_@8gOQUV23y2?GfsQY8S?Jgn zNh79!CJIl(NgSX67gtw+1AqWPpEsZR=*WvMkRr?n?)CyBR%k?nr9e3j4pkIdT0%48 z8enPDfLOM$>MT}!B_&;3TbqJH$$_;3umrM}k82}sr&g14!GBPlm; zCIa0Ra`VYguh_?s%GK2=E|4+W>wH!w{}Bx(7AWi{AUlE3fXl{je{DD#HY7o@@oyD+ z7Hq95bV(3vDpX&WjXh_4$(A_IK|2_@TIAXu;AM@tSMq-EgXIaj0dRgLG&JI&FHG1t zqOV{2lonym9v2*hc81|#@<_a5p#+<^+a*DcbG3>g9)jrZQUuJ^3%vT z8M7_Sev3~?nAc8|c*F}Q52*#O<}YV85s zp<1<&XdnpVD^){9czB{K?S=~_FR-}L0hD;R25tjd5pO=wUmMY!KadLH>P$#V0`(&_ zOrPbkohIb}0+c~O#<=d`mZzQ~Qzl+nU8umU#yin=yRy=A5Vy4ymSTBxp$uhdfu?B*(^BDQQm0MK_}#2lEx+Rn1=_;Uk{mIoi$4Ngu~{wF40 z3)DCpwcE-tG3^^%zIFi^yX)~`ARLTBP@vBp{pG5r;zDE4+1JRbC=;UZ3RI>8uRyWu#mxmOY%u^2eDTqgBt3!O>* zA5FxNmxm1jVzHCMpZ=F@O#rYC|2I~#ge-OTli;JN9Dizqz zwQu*F9`A3%u&!+oOyOHObTTC14RnC7FRz3BY#`~AL2=sIp25w%-`RWuN&Uxry)Ju2 zQYhoNo|WAu#pG`bKgCp{>-_?SG5__t`vcA31#QB1n2%ebHb(b_QR7u z-vMisK_W((5#Y%5If7QjexgcE$?Grs%4J31?DOOYVS)H#@&Rd!UemJDll?qiAp9(H z>-4z&-==@q>eSmCc0>4n>!8X+tm4p%M21FKY^mATTtoC2))lnr$|1%dIe2(}KB?pC zL}G#cWc6$4jUcg7Hj18h$2rK!m|XqnB3uSnZbb~vfk-<3@S>t32!Bir2{?&Jrhj-a z9XR3K%7Dmi*4ORtJ501e-Uy41ehlZSFvp}6EeXWRE_tu=_0*~YCfVpgz&VP@dt}LqJ@HLyUb_bVWWDD`#*+e zi;FWMfic0fl6ilSacM&aj_-!oz(AA>$YP~GZyRQg?)sAbmJ5VtDjbD*%V3t%zH1OpzGE?oL@Pryw}w?{aGxFy=yuSyQH-nw;bKJK?UWI&ic z+uq&H{=aQK1QAp#_Wfkg|4$!b5o{Q6V6TO<)_L!)x>?cafCZ==#uS1m*ArYsCvQxC zb3wjD{Jkjr7f^qdTeef=?Lzq_0-|RGKj0wz00>myK>YJK9FY`!O!yU2^Gxfd|5vCm z&9$;WX7@3j5-pb98wyH$Sf~i@>d>kxL3hSaAiFQ_%0k)5O|;c`Y%W6SW&uUkg!fwU zl47VS=qlOR*`eNVgrEf5b-M2nkJfO_y}Od{8!>c*ddVz7PA?kG%VLs}@~YvMLt+%=Xo{PeB3FTc|MMGCNB_hBMpyn7d#AD5s2llE>d1k+cZnR z43mvh1)W)e*w``X`}qESF#LWB1_mX4{WnvIAGkVgnbrPVUjowLk7={KYsN>1NLs*c z!~q9;Ddlm-3Ppj0BgpN=B6re;=1e(>UpKow6{i5=2FF8=E`R-ic^8p)$6+92+aFKGZ3g1H) zXi>>ZOXGv$!rSI-3o8YYgOJ3;|CAqenMb`(ApZ~Dg$cVrR7ko$!XSkTXf2>vPiH%o zjb`koRC(A4)(&Vr%k~tDLSB#Hsc(ZuAQp}uC>c55`kx7bEEJrm8U599EI~ctsy!-% zFTlViG1!#AhBo3R#FD)Ug_mCj8xf={OGwd3Yyf8YZ(5S~QJFXjA2xI|7)4TqTyx`d zroiv#2*4R`m;`||D`9{fx*=*BH24!kFFRUXg}=jxu8r! z9+3N*HuNR_{f#g=2|(P&@W4QQ{S%~nKaio>a5J3n&V#n@?ghlt4MQ~Mp+WI9DI9U2 zu01$YG%+;n`ia@Q2i9-ehn~F9m^ZI^3KIpu@MqQ(N*;OTPCzS@f9?0vV5gaDusIuV z0~7{}>SaimCvd~-0{kMROfs(XzVrA8J)G%@4Gy$=kH!_eW{|=lssosWk260%UkdH* zGa#KahMiAG{=o5urvKuz4V~B>4OZu9MiB+r!*pS5N`0W)-$q76!g5A``Q`ng)378w z84Fkzs5^^+d^P*APwfRKI-DMY_+&^N_`@{iyZwspDR79G=OX~&I zh!=P$LSMbI0(0FeM!!74iWVJ*!kq#jA*qLU#6M90sOQaDkct#_(;obG*e@n*PQBGRQ$q zvWihfnB@Y3?srp?gbcE|0fS-+T{id|2nG+_Qov=<^iLLczX>Y} zA1w^ZXos&0G|+tXuB$hye>>^mxHR;e2ocasJR5(i%S8hvDdJM9`PA@MSXzw@R==xD zF_GJ7m2KtAlm6>K)&;^f6585Z03TgJ+MM3+^#}Omy^YDvD(AVe;$ujXFa%fdzqLR{ zBQ90drJ1~7l-bK^8YbUAxDbwT5EdafSN66%bXp5-y@&$Dr-u5JHXHtl!uCH8XZ3wm zRWdRI6}VTIQLCgfII-})a3-#IjsJcIHNgpY+&Y1+td|EXko$F zWZ&KJv?t}1nx?J2hlPSG-kY4j`&5*#Bx^T z+_n$5A6NG8K#mj?6hvLQLI~>lYaoEd6;`G zSY1!=qPT%Us_>hl+}xKijdgBi<>EJJUV{yJ9?~v80RdtMfL5`bswYzAQB2^T#XoZ=0E! z!RNzGKZkeo)-B{0!xc=cpP{Sh0@GhXD8k9Z)7sn|ytnrZo)|nN(7N99J96VTsaJ#y z2@_wyNMu5F?wf%oI7?S|DZ;!$^M^>>C=& zH71;RE{F| z;GJVq7}QBoaCvaSG2XdzXaxmUcw&HsaCq%Ucwu|vgaT*>@du)?s;#T*UEBTiD4 z&CO_d4*)+va5E}g)S&O_d7$!v5%F8JL{I*+Wm6gG|T?SwxsiZ^-Fb+|8_w}hy zOiWby9^Qq;0^Ab}ZT)g+TZ5;>YkTpPmi;+BFo;196w+)TunC~FpJy4D9HB)gXARf$ z1nP6q?ABW(!TCU@B$pU|#e$728AReeDZIPhB0WM|>i_{jy92`q0|Nuk&9z2f5UzRp z9g9UWg0iym!NGwlD98;Au0q&*h&f4y4*|U7JjPIwdt1EA6%FM{f zXj7?=K%!!UFZK+HM(r>!Ku|C_Gc%KsL*PcZq`p3#goFghrqzv&9RfL|* zFD{M)J!VfYFLD&>d9%40Z0Cu3zf|g5+Nx@563|B!aG1yhR0}-<4Q*{%Cnw&svop@{ zfG=MJpx&N?GYxzwgoSoMPt|ps;&)sZ6=5T$01qmhekr0E3qboRL*gW27U1Lzb)IXP zSzc~~wiqCUO5y9!v4?3&a6fN_b$3=}rMtKeL(pmLEycgBC;38oc5hQtQ>$yC+ZzbS z6be@;6aD{eZL!|FM@dFT#whA5Z*HD7c#ndgj@BEM$iV>kCq1mOu;kZzzwVS+FgD9#t=dh6pEjnrHAP0=>`O7hm?;XkIM}LYcr5lteKxsen)yXLNLw%do1a zh5X{3Kk{hpQ2&(qc}s}%Fp|jA+dD2cwlPa40?ec!xxRdGd<;sP6^B<6k|z;6y7h5E zH-b=-y74~Al7uYMCa3m#VQyN4EJ3I?>Rgm0VehRt-e8nusBz4Her_j#O%eVt>oS`e k5&T=nd{F~_b~*STj*tAU$o(4YbsYGEmQj{2ku(kXAErTs$N&HU literal 0 HcmV?d00001 diff --git a/_images/markov_prop_7_1.png b/_images/markov_prop_7_1.png new file mode 100644 index 0000000000000000000000000000000000000000..70dc5d524110f48fa9fdcd6ee58e5a4617df67b5 GIT binary patch literal 62852 zcmcG$bySp5A1+D^El7iOOM^&Q&RXZ6;~E9WVP?PY{>Ae=zh@hzrXq)pNrs7lfPk$aFRg)qfH(?%YoVipXUdPB zd<0)0uCEoe(7`WXbjwKacMKS=r$4mwpK5-t&3JE7Tz~ zg3B{vGmQw=2s@f#NZd2au5U_jd~YsAy2EDND4(E)^qO)<_t#z^kUoB|hsE7%$ql~k zM^WN__V|nwB7xF>US!k*Ndd1VeM0iTUSb}#qd$)E_zwJJIm;oVegEe_`hW25|Ce4u zQvfzn|3>jk&nws<2>G;QsqVaq3lTY@;j zG!ld|DUmJ|pT8Bq;U+Z>oN?>?ekzl1$vuysRZ5#NB00nBM1cldpdH}>@79VFSul+r zY>Lh@af%BGiee6_@2c-$^WCnw-V17Ro{rR0#{D>*{ zyyzzq9@k->d5*Rk`ZUG&Mm5`*SIN)ktZ(9exd?r2FrsN-?}_Zy69GG1<#iC6?B1U|l#c9jYd(SOrCU9d zY2%s4<+RrTCqeBee|(L8Bo=SR2oXAC5ajZElnT%LFSoa%bV=xa>wls#Q&RX8={%~J zcVcKz8mGdbI5AkG-6n1x0*Xup;xj5+U4M8hp zj;`;nsE}t$No2s#)YRPF-w&y%Ko5Hz9w2dX#reRp5c4)#G z_y`*(E&JBGu!R>dUxuu#7=NsYE|yCuDl9BCZMO~W-8n6+ug5z*J^ff)D+h&Q@hVHP zrUf%7fNdZpBYO>Q2@NhcgqJLiju5l@BXd32?qYth-JY0bMvC4X4W!^lclY)rnF;&q zEry(~|86G-iOfmVn*9}ZM5n*=Z}`EE9|ZxA(z3E7l`>lN*OGxreCL)zuLYo(m)#T%NVwtoS6yHVhy{yxh$@)pE~O> z9BI;EC9+f}oSK|8bs7uDC_#1scl@g$N1WMd^4hN!zB1O<0!lX{0%cADyId=`?cOVeusjOizN`A)`dr{7jhV^MmaI;v`kPtL0untj;%j>7R zG2m@bQL3TGm^ob3p8OG3(#}0k2ZEMOrL{G+C;3l#MRZ{yt&)eU4%F2TlATSYSU%q@vN=h2YKB7j5a(8zZ&ZoE&TLYOt)=0Oq$kkFO z1nbqCdCWo5tWwEj2Ku=o3apS#K_wLxb9Z;*rBCnrqX?cGp$w-Q+*GB{HJGAEO7N1& z@RIRSL77r^4-dh&VGIhHIXTiyDK9=Trh`jcj)ssXO@w`twc&bQe#F^in4w?%gbS4Y z`h!Upx}Ayqq2ZM0Bbb)&^YR8~X9=^ivz<5N1p0PPTm284x00)g@k3`87o+FrbvB)o zUuMZOaP#wrTk>GQ7TzJprduct70ZN=F*-3};ps`*YJ>w41y@ZYwM#?2;sZ1`8&2}Cg?%}! zJ{jDGSqhQD56|l~2+A20c8`uwCr$-@ued=eYHM$2!!O;APe!p#8CPffi5aFj&WyoL zXIEUV^{uITspxFWvl*Wg+|EBpV59i@%Xbl!>+e%PG#Lu#4&LA0)Z5H3Z3@|?RvGG4 z2>$a9+!TZ$A%X|5fB<XIeJ=M)ZE9N9U`v_?(P&nm$tf=SC zc&X@K+~-Wux6Tb7i;Ih~L$0k93<|I5l6rpreEdd~N)86gW$F3&vw7DbHY3OIi#8OJ z3!?OAA0$N`FAe08*SaPl3MihWR8+D)J|Zf4qWT)vo%UutlEz@Gp9vKw$wORSUB7qw z2qX_PfGjt!Cq+ejti9@~d3qn%z@HZmb#M~dRaf$V^+$f3=*5S*Qeh* zyj=BJ=!{|gmV#34JJmwPADG4JWy?xSV~ft_5sD3d#TsNFA216nhwXS>`Mx4HZ9+5+ z9(F>Zyv|Z$0u{~T+c}#IJF%3bBbTNAU=qC?rfpYxk;eL|Eys`T9=1&yLacv*p~AR z+CHz0WBR(SRyhH#w3t~8d-*LdaSUym;xm*38@tpl$;xKaMRkX^81@l#2oh*xRqZiD zphR)6Gi5Yn?hZG^c6GfnYIPalm-(R4d*+hLPN4KQ-F3s`+cQQ)S@^g$H>otpa`ut0 zmOO0urqY^VRhE_zuoS_fe+j^3vTVDmBflJY8fvfg!JMnDbH{V3r8m<;lbhXR3*If*$qQZU5Dm5 z>rCh;LF3F(p?E1~!q=d=Oitp)QwSmT7^V7yWe;28BjDmhDBBf}fDn__)}GXy8*OUe-L$hMcF#TTk)GMJA)Tic3d(BE7Q$o z_81jjvlI7&4{&MCNqVV>MmqPq(HoXLm!rD>L3v*<8`tjQ{1K^L^6GQ+ksc_ zd3U}*kBi>2;=wo1Eud(Cv4$NJzvkB4)L+glMrmrPoBE0@uBfJ_rU>1sGEDz51gyZd z2i3`TjZaKuxD^g=I>S9YqzGj~U(-?u!#^Ip@MKX;SHhoDIxr?s;;JV`nZXf$+w|D; za<&)@U1g;T6NgLeI20WRs>Ae;GWFY_hu{qnndclT3Cp!GLEZei?ehJzBQ+Tr*`Gzb zT5Vq9TK!C=Kj}|l3mIZz7JPUzQ=gc{_FgKIlw4l>q9Dp{HI@(du}8Sfz&1_<4Gauo zc>=fI()UALDK!+zF*z>gG7?l z*8ZTu0x|=ov9VEB{hukfH72*gUXHry-mffoqIP8u{fPmROw$_r$6o*Ud22;QIA4hk z8A#>F&^%oF-2R(RgOd%m5g(r%H((ALimZi2vHBG(-D2mjsN)O(%2Szw_{QyU|DQ^KjQpMLTy+wnCn zghI@laNX(qoWIUOR&)O#crkudD4voe(@Oz9rIa5!N$|GU1E z&7qODHP-jIx(#1o{MiFNJqX}}odGUw5c%*uu6NPQ!rz|?bO~z*2Y#1Jd1cMT-1eWu zK_q6u`XV{e(@S2GxBlG<(h6~lF3fV=hXpe*uuc|kTMiIxaj!%F-d(N^1tFt-Gk&S3 zr}x;LKqA^z#)P-Xv*C>@!YUn5QXprKU>Dz0O5Tx1mjlK_~>&E<2E-Rt4EQ}!6I45w zs=08CUu3;3GNF*8)CMg8C;^QqY;Ao8gTaVA{_5@de$z-Vhe4cU-hX^azEq?0!Lr$NtY`^6crSDrn?mRkv|f6L>k{LkmX^YC%dOfQM!zdkMNrNf?hAE@Y&CMP7{{8!4!X%9w;(oSwZQy4C`#2z- z+&njE0C1ZEhaLdy;22B&a0Z3{k|+7H8Cef(x)1KXGRFAKwHT!fgUDLlI(pb=tqb$> zZ$$G(wjYxvEDS|c-CGzSilQ^)o1bjpbLE_~p?eJ8S*2zGwt>>|y~|HTB}Y)5yH$tP zc*{+<(Oi{3TL!dmunHUs!8OvL&%}1a(++%XdXT*uT7~4KTb1A#zzYVBj<);kkeZV? zMqU0~`=X# z!2|W!Xpipq7v<}J`?chVVh*qJo2|{#kvsW`qNKTngrY#iyf@IABC`DYLC)nOU30R7 zIn$Wz>6$Vg+8Qfok)FCYT~UJgy|x73qt#|m7@BXjU=2#b>~XC zVm`z^mq&b|XBN~qq@EPU7)#oZXJ-fTsGMkmCOqrGFc;5gn)}GDad9P${ zYl}%SwEi`Tz6M4eI#DC5CrGWWRdFD*4742082vv5@W!}0GxoNilb$qQoX?e+52bR z6ljqs-#Fi@suLF9Qd2T27#1bTfB1m1!Pk4X=oxr!V>bO|(|LAbA>-%x5t+-s8e{)! ztDIMUWZnPfYGa4|@AgeBKmp7IgeyY))(r0Mz9B{ZPi-67BM{>k=0+;r#s%H@whJJN zJqHI>cz<~K`9EzG>5g>`T+kzmeSj*KKW!1l-ZdzlUn`Pl2nM$g$RtjJ2N>&TMS|5HSenTUx+}Gt zLkYzoJ|4lGqvOzws(4&kp7*z=u|I#fk6JcyL{GjD$a;_<$kP6^=8d-Gi6SxfM9hD< z-KnH%r&Qw3NOEyyskTFHO0-{YBLYFhEEuKRUu`hfHGg@mRamCuX;x%hJJ~B^_{kp8 z)IpU`Of0UqR}y3;U^(ySWpt&aJIMey%MGe1Ot;J2`pF#59PPPV7~ym}CIFy^@86XC zFYEmgCtv9k5D<>KZXA5qu}mZYsw{hT&SH`8x-)JD;`8&WfBTE0gn--4mhE)I3aa)) z!Hn&X-%-?mdFRBMpg+%QNG6}o>Fz2wl2{dw6vd#}FzxNGU^ zW=f8Fpg7@`LEY-na{I}}lB2GGS_XG-6)i&s5YJli6_MUC8UvWLyu1wPMU93uD+i5Q zj31kV!tk5M#|Ds*=7Q|q@riTLDg3_X)?XZOeyrda`OunX2sx#Z78}-R45vO0GnXED&p zYv(6t9Ea{R+;{d(5WVD;ZW&dAJB+UnNL^&rAJiL228LrSDj1U)TBDO;Y+&X1fIWzd zKLu3!`*%HePAjIA-ZVB{+=L?3|Kn_GP!}+SFZu#-(4>Yg)Iv?=>Z|uAO0D?Ub$4pko@WPYYy9XSONqP-xP~ z_9UR}dhFqCZNh-AXw7bU^3lN%J%aW31M_Sbl%KIdt0B>wf}rmf*3^u{5A6Bvs$x`> zlt3W_lCfqM*u5vR3fkN1K0y7lGS8sfIK4K)dYVhuU@?RPXdyKac`ET+ zG~?W~MulNDfmF5nu7LtWTSVzIMaBD>pFVveOBiXe|JjqgMyDYt`d$e{gV(o;(`h~Q zW(oqs{+|VeBP9&#H6<6+K(z$S=F_{SH;Q=r9BlPj!=ox6N6IHP%iH7iGwmYgoPz${ z`9Ers9bybE0GK}oFgY_bGq7eKg%YMi-`O8S{DuSNWy&xO>67^_J>x)(HyA)-BPQTT zrkh?`iuv{Hjp2f2bJ1rqy4Nwftwi4PehV(oXSRQf=^x&Y_B9Q$*@ZK-FjL@wI)Xf9 zJ-d6%tbqUK%^PqppK5F4VTLK~7X?X}>JR;r!D`eI%48s&wiAFn_U|AU7nWT1QJ|3k zs#IK4gR=nh$lc6Mr$PVvrjaPlwV8=wBbkz%oUQ%!Q@kY?E(R`xOk;ed+Srh_=99^t z9L<0sxbQnn&Sa|}vkXFHap_MuE8t4Th4XYrVcDeyy^xvLjIF6USnA0PXJ=*kTZgNO=UPf>ojEilI^ogNy!-0{I;2d8VRmFtxqp;Sz?~}m_E-XYcz1fakQ7O+=(;49v73S8~=mZ1=YhF~Kq<-9uZzB%Y-?LP+ z6`Nuykki&yZAN87#WVQ33~*IJWhK^>HKD*yC*;76z+{?4AcTV+smWjkWQ6KS9xT_x zx}TEr!88Shg}te=lEQ6?Yehjph|JN!YRsYX3`jESiDPeolrClwx6q(D?9FO`V6WEO zR2qp}@8r{Gn-yMP|KiEhr{ukyZ^v`WuR?2exD}%17=!mpHKSn?s?FNLta$Gg2ckNR znNlP@JOnKD`RCKMwPiRnTVJIXk(J9W?klKaJq1SsEES8nCKm z_J+^pUuKCRMXmIQXRPk$hpRt?FK4;83lVT2-_EDD?vQ z4+@1UlW7{tXf3#PoD7nGueXqsR51~x;5TZWvWCmd*K};mpRyYN^R-3<*e6H=AxNTt zrX{i}2Iuik%go1fHXAwW5{9)}QtDj>EA;in@HwI8gqX;a;N|ValtyaNXU@q6=&)si z=#-L@s$o5w4Nj}$CLtyUg90mQIy_!p-U1+N>ag3Xw**Sd87w;hE#t(K0@NS?=tGN` zOA!*&@YTZwoGpg0eP025nnib{kVx0LQe$Hvxm~0k~vyU zdoELJW{=m~+Z%*Tz2jO2CC`O=dcb>yqn8oE`8AY+25wxgOC8X zi|2jp6w~3Dz^ak62P2I8Cu>nM11)jPGde}xjPfQoB8x+H<)1;z-tEh+L{Ct$SV_{_ z2JQBeOK$RO(B?Z$k&c)Q=seqMia`z4tug9d>GXlsdQ`u$m2X89gi9n$?DT`Dlu@cO zvK2%8qcJ!r$Dk`uuUA<8It`>^cGEcw5x(nY3~##(O6!`KOi06XHAbR^;ZAr#kT!+n zP{6Go*?acbBP5YdJ=)b6aRyJr+TFGND7b*WxcJ7@6ah?Q9>Kh@h{#C8C(vT6!d<>B zl1;u86l~cr*MMSaF1Qs>akJPucG_LK{EpCcd&V>MBZ~Q2|46tjcH62oBvJ7HQQTe| z8fKpe!dk7py~#kx1=ElsWer=}@^BKLHR4S1VuNxHXcs_+I^9Z>z-oimtO}b~*)VN` zl%XZkJ%cscT-x_7&$D7?49Y_CDCutoOO}%2c7*L1xq3{XD*`D`1_;prda!jhR?PQR zHHDz58F53Ke^$s4?749Lvtd)Z?Sf9!5>PX7C<4v?c%_P#TjJ#jB zl}6C`?{M{)cv7h)BcP!p1v=j2(I9z$#955DEAZQWS`e_dS$bg#FSZrn_zaQ!==pQ* zLqzIn7Q8nSP|k)^mW4{w#r4z)Im4n4o2Znxfb;5BH;gK-rLE-f8Xtt|W0v(aD=}GA z#rW@tto8yVY_a7c9Q9Yw@aQB znR-uwptNMv=X_M1{VtayZjRcY3M|7396HZ+I!?z$u1*pHVu7lDJGOcoXR`{}llk#V z=i>+kVeNnM-QDH?58wT`?`Jm_jynpt5ts+hSelhA=B4tnIA$nHc>X)?>6VLPY|VpY zK0G)0SdM;qi{`GB-@g`@-#_*A{#BDF1&F>YK;Y(`LGhc9oBY-gdvxRgG*Q4Hi@Lg2 zGG|9h@UdJSIO7Hf&ksMH-P}Fucqjprbi!c163P57%3ucf;7^u>mwHZzwb}e%S1=6u z{L=;N@q&il4Gs?0`Q5m0KE%i4eAclQf%5}NsMT9t;pKj`y4S?dL=(QnMPb2KW6f)|~lE@fG>#Z4Pu-XY_`HUTC_n<4rCL~yZlmauPMwa}BEK^zm>#TU{+X8Zvn?-#9 z9LC4r7zNxB8~MB=zc_Y$Ehjgt`C82mY}}bY7(;;;1AGRuWJnxEXVN2(M&7K_YQ?d2 z^D6kMb^Bq?cy@+~|FbgH{J~oP^U8+HYNC~YpVBb|sOiejG+E{hGK02|kZNBcAAO@J zP)iYU9!7%dFxD}AJag0!J}#M%I150EI&qdSE^e7#h$zf|pm@!VZj6g->o7>U`tZuP zP;G9rrX;Lyar?|&!g6z*leNCH}lEZw3!Zi@33C-z6@Ur zGeP`eWPEEqg0JT?KrDrp7)Q^2#bM2ZCiGsCp9<}>=i9Lx-^9Ve$;+AiqSNlX&$kwp zP2md1a`P-9y#?>gZfs3zGfk5lqCj>IQQu8vdtNS+5)LHXmK1Vymt1ZW$J=WG{BkX@G2-^Ci z1_cs9^zLlN^LKff(h>lEoADw?Jp`(vL15gU@!wjoH{WM8)|8z7mKvEl8;x+eLq`Ft zELU6M!B~2BbInCmOCL>;GXOm=B7H=1%yRPl?wCAS-}a|PZ8r`ruEc2&@*n-S3@!3A z`Vk&)A0Ju;e9%>);RRd~Vedz5u8usO|GA#HG)^4JhombqT18wz%?su(wN+v&^AD|0ES6P`>;JzWQy zet1R2)z)+IsQ!qjn@Mt1hzLJH0lDfU=?Vm_Q`PUc=HM$JAiDna5CWxcFMO_vyVYYg zQ1B(9iu&m(Z(@FNP={Lb>F4jF%6T&g>JUMP6NxOG+)V`SF~>e>)|V-NRWB;Lf}AVfk_w@6fv96lq#17Kl`xT&7kmT zt)NTdo+gs74+9+{D>{k<*08VtWrW4RY3beP^-oI#Q*IUd!9|n!H(N= zxO%!bJ}qzZ1uwShurCO3H$w) z>fp(~2{dsV0YN-nIvL#x$DNv+K*>W8Bb5f-+S;1ii%?$sy{D){By%7Vv}tzA2SfL8GdFu^qFV=6XN-UotJ}?GsTkHHY8QSjJj1%8sFzX zcQi{E3&oaeT}{y(VKt9R?n(f>EbF;kvAXU~5(&yojD-ea36ou>1FXR=4kT}2L}h0z zw$zTNQo5 z3kMB2bzPRzjmsFC?PvDR8c0M73cGLQsD?M#OcX3!rqN(K>bt>n678mke=5TEnorvo z$+h!#-pQoaI4&+N0ZNI76M^kf150wMZL{nIO`^h!w~<=_Va`}@`)DHn8f9_`MMXV_ z-b|sZWX3gp)nVKNrv2h_T;%J(rp$AL7q}AcW{8U$|Ja9|af2lzduduE5X$##0u`y4 zfC&nWhXG(TU#NkVx_vW{S!fH}6#Q5bKjEj+FVJ01PxX{0zqr^;ft_e;;3f7c2^h_J zYjYzWdupab3q11#*rCDN(h5mVi}+*PE_sc=I<$Oq3o2}t7YAQt9H0hD$Le@T$gn^A zOc2&o7r50bP^avX!H3jC_|szl)7-mx?n9j7eZ3YqM!)z!Jg2q5P50ncM{jXpn^Pz& z{4g`ghb^{x$BAPU&}owoS}l+g3aYEe+;ba<)6In#QbfD126RL(bEx;4B5xyPicI#p z_Y_s#sZH4iGx%vd131Si4QYb%HLV{y2s|oYYa2 zc7t8!n9ff#E=^6Thv0d|VAE$heAKp$N-;Gsbf9!tMSVs3X?Z?W1{a!8+!ThfwcOCy zeT|)UzaQZCQx+Y9pd6Al7QA_V%OLNMtoHI50ER$+H8(Ru1eO+V#>S<3e8wY$wL8wW zr{ahG*8+ZOyyFyJ_P6vs=V(~zIveS8#7b<^IS0b;CJ!qjBg0?QrZ5`(X7cQ;BeP+& z!(hrJ9MSIRDc??a`WFyH{tNMvp$;$ft%bu>q!3Bjt}9;)K>_g31l>jTM51P`K62#aHWrz!R4wZvxN=CM}W?U zvQ@n_6L7D^_+q=+JkdXVw>MkEnyVs}%BMWFth9c$$dN}(IJ)xI zzv=Vm49^lq)?E2z%tN;Bb*F^~4s&=aKGcrJzDuj>oX99nO8Y%P-L)8RgpO3>H^E+k zM1345TRZ@VqC#?AU0q;OQW6pX>cxB*nfl7#$=*2g$C|bkrW2R`$22Sk$A516_^!s@Oh__LFI7o~Y*-rX)BsJf? z#ckWMSTY}?u8fTT4@ra|1tH;`Prp@4<`+e#^TIuY3et@~s?-P+-+32L?zph-)jX0f=E>1`hm$XiupgCjl{CrT$h_YtYoK7yRxgIWiJZ zlz;DW-jA4_o!Qv8-?>Y348)*xQ)lIFH(X^VE3V%nKQ-Oq_}tU2m?ox0bainm`szU$ zubv&DZJM$k6gCbhnPGmjpD@OqhbZTI=jF=FU@2gD0<&`9_t4=}?uFVzdYgdf4;Wj) zqv=s0Ccuju+I&=cgmY!VjMc;>pSGyCK!7ZL!TbqWDHuMX7M~E4%W^N%MAPwTWrdRO z2IhegOsz(K(>o@`akO!ACyF|&5gj9sWfl6q zzsnx0z_$go=|mQ@F2F)Q{T`L=fgYKeo}-B3qSN4_7nGEsNOU31`y6*hgomR7rw4Gj z0k;-#fu%5#VTOs)7HvNcJt|1&g0Wp00V)#`5s3l5E&77zz&Q0@DSZVGfdo@(=c+`Z zGx>VJpe0bF#|4{`FR7=8q&sqc$pJvnzU`5WUS|p%V?VyOJAX^cp#rIeb8XiV{}VLP z`@J%{UqoBkak!Pq=%sE^-!FKXt+cafZ5SE)rjIWlsRypiR4TV>hC^fYu8R_rF#!vg z@|iIkzQnwip@G2@Je0q$6@RwQ%*=r433w?(5io0L=2sK>IjiU;fZT_N81}+3cb%6A zG$$+zuyI}^6oeq_Du-6}Z@%hoZy~nORSpw<83OITWvq#$1Jm)q(o!a)z2ga!oE$%f zS{w-lrWXNw_;hSJA8#|nasTX2pcttqdh}1T&^I|Ps%r`6z7iVef0`1%(#$l93@up# zR^akT!Cdxkd@)+xK;9Mi`CQQ^yB7wHCv$L#7 z`7E%txhu3Xyxg8Qs$*MzvoH>ZDVljekL2`8zWm0p&DUGiOsWMHBCeg-kuhI6M=?%u`b;U)hPf#flV!$;; z!q7Onyi?t0_F>f@gg*IRC+z7u8aVEURa)A7Ouc}QGNdTXFvv^KLJ7JFj0s1ZP0iW{ zRHTVy>cs+f^NN6BDn4I|!v0V-9!6Mg=^*Edi@E06Q%Se-T7>bt zbN(;@#~1kDEn5CHnoL;(CLN9P)L}{>B_0p=;4)R)64;j2Pu$dbe=)K5pF0| zxOS*V%}jYF)o3J%#|MzM)7qi5`I zYuxlBVeq648xLl_O7r`2NAso5d}Ot7D1U~Ox%I~__fQvzD<`t=FCxT=3v{n~Ncy;u z@WgZta}ew_3A6GR!Rg6Aec+~bZeS;OXaOY!3~zwRm;U2N(79z&wmX^B$b0+XF}aaI zr~;O>(eZIIyWa>;JRJP|#DLss6y+;Q^_KI;xkP<46aHk%u%#2w+O)LhhvU$9v(iZ_ zT)5KB@fzHlE&Yyc=?Dvuz8&!5zkNVSA`f^wfZJ=!sci@FW2V*>N;3o!f! z8DnZ@R=7bHS;K^`FZ=iSfvO8mCS1#fGT4Uk&v{qLoMd3LBC4r>#1;elRQw+WoxnX+ z;DDP_L`c%zoDoi=V+W#_8KOT4WBt#cMo*XmAFjhA^T+^gn*76mbpz}m z6NkVc4W>~4Il+@7;!%T|FJZ%(CH)_P4`I(c?uRI2o!`FSGNt3r~ z`5~s|Fv{jeLrB8k56P7vAat-gAj4xsxLeC#95yuS=#O0fnPhCi#N@A=;`t|EAH=H^ zEV}X06E%4}H#@?r*bcV&3l!5@KPS(sC-b)AN@a^>Ku|yN>?kPb!JD+^HE`#ST=+`` zqH2yf#02N3UZ?29QnWSCdud>c>bmSaUI%_0Nif{LIqpu_uW83wFbNQQb=CVc%Rx^| zd!S7G&ZgsPBW8DZw+C4DPNow42_tYQc|O_L+V+J!eHHmO&uh?~ovZt{p?tSrajn|X zK8%@oQnLaBuB>MSon|ZlvzZ!TWL>JeZxl8?KBp-gdUV0N+ODg*O*OotRVfvhKHa0s=D}1fl)m6vT+hUnnDCdBN z=UCra7L@pFLOK31((uby?gyk@=hcquL1*vBETtanUIBuUC*;r zg0DND;RO8XH+#KZfYRDmAvVAc-9Zu~GXH#4o&jqyt(FwfF0CZO!z+Q_Pf60kElU<4 zn158%(RzG{xzv_^O@Z?nf-YO=z?N*@J_hpbljEwx?JCf#fpx0IA3J{YPf~&}1elU) z+K>9c7V4k3EY}r6Tmhv44T+^qR=+YYGk&T>tnC1D&3cWD zgJ}uGL!xJGqE8t~xn8skz>*a@9&`i3E*}1xC_30-|4ZUqvn%Gg_h$+WH3?M-_NpqU zf2Q@8Lq`jLB365Mp&@LKxNVC%$atctl1PRQ4=^e)a#O_&A^B4jreE2I;xMgvP?O=} zVEw=I7X7f`$iQaI1TH(S_s{h!M2%S@*MwX7ThzWrCN>+m5v{Kk5y=F)zEBk+gOUZ> zVBaR-uf4`R!^epK@@>{g_S-b^FHhE(2ceu^EN=oH$=(C+5?DQpZp64|Qp~)~l9t@P` z*w|RohXf!B07q1fNr?Rt2#+PlZpyb4c~-%#WV7G;;K5iCg((f5CG+$)Rm9d&e$vj) zzqyF2EnM}Av~!*~iI2QScRBh!{aYBW9adulRW75X5vrYdlCX)Ti0nbq|ChTOpthnJ9^KmKG2XdFb~TYgf#~kgce#b6XXH|= zs}Euqz)MHG%%3~FMyoy0-TZFo$&Qpt8e$xp7NVMv#Lnq4*U zTxt(Hvo}1LTH123@ z@b4BsaOOre!u#a=C*~E0@1ael!3(@syi{$d5(EV{;J{>H-UWSrpbhe&STmHi+%}H` zf9(Tug~7iOpPgL&+r!m+)SU^{wTeo*mW#zPL1%L|9BPMw`#a@7H=S0n*(?o7ah17f z`Qh&Ja+XJ4ap#{RA|Rp2SK|z(90sS{YyjVpdv z3kE{7Qdus~I#g59DGExQWos+}CqLWV!k!%ycZjLYsA_tS*c($)`JQ~RrbcSQ5;IR% zo5(T#Z1Y47wU&A%G_xmUW(Ivvls7k|^hAvhv5i$i9kSKnB{d`6d}^uu>qpG^QdS;uZgO3iuMTf*f+`wT}JQy&8fBlMXsV~b+$Wt9PZDVGP{~%j=>J&CJB4YO;7)Bfg}*ki zNJ;kxmli@#DtLlU_z_to12Hiq|NM~(#H3<{Sg#dIgd*VylZ2Z45moLtQ&i;D9zHE# z{`FKed+yJ9?ZjI4YR+rXys(;u-W-Br^YUZ8>GMjvtoY~o9icO2Tx(yGbaF(0#M{+4 zyYG41G#J!~Dgn9W2`aFDV=3MbY&t*cKVWYAQDsNpOe!X6mmsw zUAkR4T!>|E77G^AtR;R&v$Z4&Q9vp$4HrTyO5i18Z)aew#hxbxTB-q^Gx-w+LILiv zo8Q9I-o)Z+0U=%7H2nz((viUC5vmc2NhNfZ6!KKq^W$u#i;Udc!sPvi1EhmrLpE5a z)t`B;M&I&w+&JL38leYLLkm04R8g;^vAm+w7G`)UYr9{?bl>m03I_P&x8}rtUNPwL z=w3qZ&Xm!nXlCE}`!-Oj0#?-mN8SEdvxDQ?^XTMXhK*wxps*LVf6u}hpE?!>OvpUNdFy+NEsM3-E6 zJRmJIm8i%9&K2~hvAG19dhd<<^TFcb%|(Ee1A;&yP~LH?F(Ks_O$JLu;3==n*Kqybz~dHG;0&?T66ArG2|!c!>-)%YfQsnkLRY zdG$ZN0K{roSTi;OT{Sqk5W_;&+cI2bWTxjqCod$%w#%FlugQ)?uMJUrU(j5g{vA4M zN*_jEGg|SrMJU++*v^aff>G+p7G>?1Pg}99zujxAvaKqKOOC3Jj23Q$9R&r-HfLF$ ziQl}BR$_|0wX)<;kS2pKlDO0vWR|CfKhMkT4S9-VVb6}Dj78^IJ^i|hor(*2r{eTN zNtv^LXCzPL=+Z-7uis!?{@-c8Rx8Ft? z{4xbTH%j%2*!N3=gDQlLYMOxfD>IDpr{~sNxxRZ#XlhL?&mPVAv{CIZ^DR$I?tK%A zeuV?J7xrf-;kA5~jS9!EpiA)#Rc6j5=(oZ@oJATjGjnUpSKhTD6~^E&DQ@~Jq4u6U z^KHb8R9`d6p@pvWFPjO^xy2_eSSr#fVs4dxw*6A!9yod+CUyah1oT!5vCPcGg8f$r znC36Q=>%A>MS>eG?%L$+*3jO}=e4cj3pxu@HNu6KG<%%E{Xk2u|<2Gi&Ey1E{ArEs5vX+>V;esH|2 z<||LpEJ{hg7srSqJH{7{>+;OTYrJ|({p5TN6Nf1{{%^Cu4#h)#SGlnX9lY>w2@roMc%}w>6-j%`749G`Y zNEoq|&F31*y{34_03RfihkrDvXatkI)Pdfa%WbS$+LEkkh!Z37B|oWGIqdYg55dWz z0t0YNG&~aL$rGA1MM+@UjkjhdWWAzDFShR8`VhPNa3`4~mK|0EF>^SPTK&nfor#6? zwt-u>!MfKV4C75)A%_xhSMu?NffE4sOP{c-Mcirnox_TxD|3)LPbFkwJQCFd?;zRB z&=MSy8CwI$vJxN4&@|$!Y&LO%4;#M2R<9$Gdg%jKGP#!Day&T}?t9x8+y{MVeynh;87ANO7PnT1QXOF;!0uX4nMu6M&_ zvMJ|+p0v8Og#|qsIeFX=06XApD>yL+>J`vYHRGA}RjY}lq=atok5$+DP*5rq*`aiq zU%qK2KCBG83VY@m5@)azjJky`jpL^oTYl+_9olU1z2#7dvN_dAH8aZBud0!4GRC0fJ-R>H5k z6r#^MmQy?B&qJyu;M9RM1+H(Nl1$}Mu%8GF;B=w%xFIYSWGdcsD4RodtZbk}U-Qc{ zILoho?rdDm)j}*A&9-zKY+}-CO!I@!i9gPI)z}D+=VlM;c6$>$b_oTE`H(p+yIs3Y zR@OQUjOSjkU>XCSQB|?2^_Pn(0gPn8$$Oey75e}D#}TV_vTW5l@au+P@v9K8j!UYi z%2@p!(h?6ay-s#g!r0?Tl)Dy-exTYZHt({AvyTAPYWMf=*KgmJIsg5@3G2H}@P~A3 zBxacqDx>}4x%N+*cX}$k7NHgoO}AI|UuxBfW7f~uEI|(N{2y$cbxhlRxb|@?LkA3J z8%}X|cXxMp*WnHuGThzW-F3LTGo0bhkoULuIp@EVv}r<`CM{pN<+?t%rl@bUWKOMU=bh^P?%m}AaWCJ8b426xP-T1C;E=$+i=uq>_2!3{?tbp*)m>cCD%8*d z1AqQNO=hGH0~Zk)43vOy5Gwmt2Q(_*qoHv(7M6@FiqS6YW$RM!m7_(##xK}&r~z-o zpanYNLx5W`J8PsuBSnf4su8D`2XsFG%g^23e zw#g(eHkcr3W|7Kk3d{%?fLx(?=h%USDT7aq{?F;l!V(Nr%YY&zAm~abN9H4dEAjub z8|jMtmisC8I#9CR7?C=oTLk_5HeZF>qE(PTG_lkI4RhdwNTocTooTnP=Fpi)DF@-H zvE3INPom`+6p>TN%MjTL*F8}tTX*DiNJK~yF9v?kQ=!}tsUM?tQ1zlm&x*(&Hp0b; zN`tD_u%t+YQiV!?X>lk2Nw=4JTwQ{@8l|b1AUEB+5e)j({nYVSnJ%9)BOsUu+H^xb3?aNqPv#9bW+$7h<&A=02QfU4d6W$j==e#!dlMr$9Lq zkbTjXr~zif|4gAjbm-Jc@=WBooW+KV`3m!^=^{?!sbcA!vc31l}tWr$KUEQ-;~Io~lGL8P;C+QJ?b<=E8Aagsv&_)Fp+1*7R8HBjSXv4s<>p!r9Q}=`5Dlt3 zCC0X+ z{f(z@bU_*U{|=(rWF?Q|!jEy1C{Mq%$`vq_IbgMZa8^xAxnhimgn)^@{UL;bg0gEd z1aWbv1E`*Z=S%|O?bmN4lig59b)3yoH7~M82ER01arwOATYFG*t^8mGpSpWmyT~d& z*y#@ZuNBi6NID_>-yczl9=3>kj!J4iy$cjMXnitGu}?xu7T`cB;;S6*a~TfB9M{rF zsRRErN&zB%plwchL?;L3cf=atqK@Nol0qKfcqv4VmBWux(#GA}pog7Ukhr{~R&kbv z7SDu7FZZR%8Xw8D@mD$a-!y7;Y3>JFPa7Kl(?I_i?KaI~(fA7WHb<^-f|_D13nb|6 zvq}|=k`uZrcm#H+58M!RM$z+)<;mM+RlT!^)s*R%3)M~AcT1iHN~ANHO(QkAH4+_e&y$X{b5~g+dmS&cA5QtgJQF# z=iT=~=jF@5i29BFG9Hs$=uYulllKD`yuY;IhxM=@k4QqP;v@!`YLQZd$Kq2+Ef4g8 zDFlZlE-YMfL>g`Q^`I^0cX^Om1wHN{lcWxJ(QNm#; zD@Y9JI$8{ee)r6KfI7y)Gd)_b-6ie=k>?B;3vRc;k8`H-*0BP;xiDH;#>OWCT;+45 zdi!?H7#o~M28w&JN1t{Ys7qi}1rsF0*WmXs@?{4ZW3e$k{L;W^HmY9efriKT_hP&V zEQF21HfW~P$bxUOg+ED$&;=9VJY`#GYR*G*1gM-q_`W&sM&{$FTb9RF1aWnHp1KM= z!29e*IPzRe@P~$C=U75X=#_xWV@+Xk=*=9ib|HK(gY>+wzO(O~x5sdHMRPnGhYRz= zBn)PiH@Hgr&Xv}QF15Or?Bb4xT?qI*qd(rgMptV~Q(KS4-jeyBgZI{MyvifsQ$)%lgKJKW zMF8M3P~su}qa4|{5W%w#cdGF~@gDC!y25kOP|Zjr3s2w9IAL=hP9urxcpCK)f&kj# zleA>TB8jk5{&}by#L{Cx>|j~Jpb{Kaos!p&J6{8L5L!W=)kqperS<*-D?|$f8jrs> zdgykX^^(K!wZ`c7#OH1lG1SF3=-Mhna*UpS)Tx~r@nQy^jNqax1{{SP+GTIDue$2C zYe~4mQ{z4FZ_wLJSB3DFG;uE+gnceI3Gd?2IoH2akhVe&Zq?Bj?Hs&cwQ?=BG#A&| zyH5MxZ%}>5$yz$Tr}8x)-SlywMm|L+!7!(047WNT0_cOE5~Q)Z;5E&ORc0VW3e`@o z03Qlp0vF$ZDlgC?)%P=d*W>Hl*qPFSN*b2RJ=pq;LXT&&xu}q16V+K2O!xj$t~5Hi z+}B3F+bx%0s&8b&$G?&X>tWuPUu_9X=6a92eIV>0YT!eRB$MrEUF(ISJ<)m=2++2X zLXz#YcEy%qAld_aqC;Pvgy4C#fzny~RGHtrk= z8`mMDy1gDpdJU%M!ky3B(Z(gP`vtA*eyd@Ak4eNzNn^!JT2ru2#_A68S{914RAdD3 zf42OvsygqrfPw&xdX>R{5Dqa*0gOy*^~zrnz>VlAcdTXbFPaij3?`Lq@woZ|@Mz4< z&3gfP-XE~cmlUqyR9R{6YDC=OUDka*<|!r=oV5hKV+Ud~>72o2=1%f?LO0|&rl=&LNnj~!$Ind12isInjn+gV z*I{ySrIR8yVD^S=Pv9@tAgcP}G-j)qynJRpEJ1Nxzt=yU3px=zWFAtKm$etSAHpIt z^guvEeG2qIN9`gp>Wp(cYXCHKA|GH=oSkvqQrCN{u+kyS{0H*vU@T{@h;QD>> z4)l0?X{tN=j*cjScxSt`ILinC9h|Y0t5+X5Xa6MttN=@GlYjFQ-tXC5{xbgh^9Nkr zd5;H3Nq^T46B(Z_R(fn2MNGGZSaPD|DO?_F=-#eTx~?mN$2%t3%E07s6`nk8zaajM za(*Ur8+a18(yMprg74^~PgEUc-A!#LD!v zFabTn_imjC=yN`*ydzOPyKJQUqiWVvBb0;r&jqmFnO{ypqwOh+`p02Ep=;iX4GciH zI0gcbZ#+AleG61)#E70|=OJi}Qup-lPD$@Ke$Vm=W^@M9xUk-?BsCVtJj-%Xw}Lkl zSzG&Uug^*e_+%q}JgytN=KOR0?DE_Vk&hQ$QpI|~chXxy8whD!wpg|+Cvs%5Q`GZ? zD%7=R6L^4m43d^q)0aXA3td8Y0~}iGK_z`8R*q&bgAq6B-*YuF|x6V)oSde znrH14ChJ8O;1UrX^aMaNo^Z);5*Jojfa+%~9M0D{0m|PB;3E9?s?e#Ye@#An=jFm; zdv`06^cWyN*c79Dz;F|}8QKObgBgG+#V*R&xHUUK@b~TZ?~nJ>O2pD+hKHYKAyxgw zT>T%&RPfmTcvw?Csg};>x(JWyHzV{@wS!Byq4!IQq%-r8G{J4 z9jGr4T(M{Gs)yXM@yPiuN|WhhY(>uav(;2V7@y`8Lnw&^C}qP} z{;n1p`N-^f;dvQEjC`3o)iaYHXjb1BHuM(Sy9*Zbx{3`xp*yiys`Gpoky7JbD(HO} z)z8zkW?bZoz3X?1urg%RecbBS+zaWZVp#f^Y5KM8CPg|qD7GNC+HVqGru(lxop@%4 zw3N)lnDq`F*@e@uNN5=9LgwA00cXk44EJzH0D{en=L(^V1(&YA2Akyb$Gm3@S_9^q zwScZbPZSl{(P>5MbXVDyZHRBBOAj|IQz$#^yb)x0JY8=ruOXaZR=Ys0QeG!ovIt2j zit^+d(fEs7?~o_0c;3qFU!FlGh>$nqyuH=0$rxL$+tUqKn(#j7TotI)bFrfEgB4?s zo?{pQ1J2DeiDpz@Utiy>4G>HNYhsqK%j?d{&hKa8o-T@}qd!&f_SG=4eCD>^Nw{b; zgZ=AMHnYzj0_?0Z<weqz9!zix#Y;Okl$fm1vk;Iu6^s=_-07N;ouKcK!E-sESuf> zFZHf#thb|>H_9d6q&Rm@lR-%GN9=7S@GqX{uX6kV0!@vktuc8=ng17b3b{X0OnsUF zhD4ef@AZikO-w9?qV?>1r}ORWgrpdnN7BTfq^NZt4{rDSKfB|(sM|M6-_aod^u}yr zmQD9gd`(|P`rD`4v(*nDgO;E|qZ&uN-LxBI8gc}!2Fal_mQ9xbm}JV4vXp6{f<%8Y z-iPO{<<6VARW_wuwkW4AP&$wBIcF8Vd?<$lq0V3vc`)R`?p0e#E((D@Ud6J#hu)9= z(gH3si`6hLY}k76g3)f{x4AXHv?3xE6Q}0ksXqVk`Gz$>OHwuQRY)Y3LwGxsZuf9* zTJl>^&4VcI8pboLnFkSZW+&A+Z`9>z-L=2(|M;42-r4q>gXjI_#(Nu@xW`rZA3O(0 zu?_V+c3>d@krb!hdP^^0lj^;_Wl1u<;_klTCgAtY0B8Wlo6XW{qy`6xgP{?#v@+c5 zrNJnI7^PdiVbdoqv@dUDKRric!Js*qWyviZP*>UV9eGUG3taWBswLLT;b$JnyBChc zo28Y`dw*9hKA)a5R;*n85%uBnD8C;#eh}K-Dg|vm?KGFmX8Eh z|JJ@JZLl7Fl7On0w9*{2H*~|qzL?Gx29SM}PJnYR%i2%B+m~0pT5lrr;Vka2e=13n zEq&VB60J?K&<4#>%hkq$MNYH(=l3YvOv@1!N+8L-$iEyx0Iz_0-#}GHs zqLmlQhVDCT=Inlc(<59&C@>DN4xkXwLTZUHiBZ7`V{iq{m8qaLWp>+13&12)NL<9YU_J9*C-IHc!F1Ce zmQqR@;77(PWa;IoSj@VZZkK;{kcQRTCjuVNbh`WZN5Z{{1cF3xS|8sxc`EK>lLWl5 zXE7ykz1?A1K-+j(!KHY&JKT@Kf^shYAX;fcux#7VW#v3WLs3NJM7>P`JpcWf8W(z` zN;xSEcVeG(yr8$2Mxnmywrb^7mAhbXv+wXFWGiv3+{aIMoNd!z)M^}Mk5TYxmad`A z@U&ws9L-Ym2^n{i;K4j@VW@f+I}=lP&L_@261c7Q2w4P!Ypp0=V=c2qRTrI=C>VWD z+YJWqYNT~B6#Ed<$dfS;;Iei>NmLm ztPwn>)ALC1_S7^rZA@RlS?@M*G#wvHeq*GPiZ|Zlf*H|6CE@nw+_BQEWdv||!ND#P?KzGu!NatQPO)j~b$v z9=O~@Jgq`5thKYvG;F>U{vYOhHBYIq*W-CKPc!^&Qv7EI+Hnw9`YVZEd*et;46P2E6|R=(ub;( z9fqpS(`y58Qvb~e==_F5b8%nUX%GTzri4FE;q?+}d22neMQ#fdAUWs-mW$WhmADcO z44bL`I4Im{Ta2x|_#{zFBPer2qR(D|d{SZ;Hs6qWJd#z{E(91^-oWqj8eN5!h-rEO zX7~xng^GezdrDj!-vNt~0Tr~Ax|Rex#7@zNN+q7^dxZf9&?zxXtWSgG&ZM{1WhYe~ zWc`%ziwHC)hQC`%x9n-8w{|RKMRJ$%l!AiB987lST)bcF`Zerf2wFbD$nS zxe!7TGjRWQN6$#!mJB|uJsBsvf8*ER|JHpu#`vLW&cx&p%WFwK09tRF%+ofCjV`c_ zvhgB$IUmL+ozmm>?Nu@Z9*@sW0{Y!6%Uu~=Nr39j$!K6%j6U7UPc}GI0_21@&bDW zuN`kIc#e&tsPuyjD2+}!-ivghin~h@7C(zSFIrG_Vy)Uj630dA@XB0|^GhoS4`>S1 z!|f7o_0gw{Q<=0P-a!CH+E!$JPB}(~d21xCNG*}iI!zz%&4@yy*QgLw!x~VaXOc-4 zRlESnx}*0qRGwfg4ubWVWASKVoZN0{YqebI_!fDr5#{EM-~cJ95US|J6yJ`hQhIkj zsEoqAU*5{xvr}8xj85{x$c>MYP}^c$WjP3jSgm~Uq#maxT{i6_TA%S*59?6q)b2;T z-h`YGX@PE|-=7^E9iTbbl9Zjp8<6PAiMs?%U9brv4&%+=iZA(j$A%UlYc$8#hvlhk&HS z#z$7SGol~lR_i88ZI+=?S}&8qQ%UPl4k9x_f;a>#y#1DmFgrSn(n=W<99cYd_42~T zb=f2Q^6x~I;f{9s3Y?q=qNiZ=Kg9M@@{y5dfd)+MsXAxH{o;}VVEmd zNN%V!B)}ffrW!-mOW@+6OU#f`qH!}ZerreCZKZi4#TD`G+eErJy;`3tq`f)GEpb?d zM2|N&y$Oy=hLq^DWPBOm*pG6SD9@*87%HrB zDBS-RZyK&^$@GZXOleoPv$9iPdsz^Iuhl-Aw?H{v5k%tz-k1?|Lf&$ubis2ne9+I% zIhp+07ulZRIiAo#_+zHBzaiB=w@^Oofu)&dt4$uPk<^$vhR2#)>orkql8nzF+=`k) zqmpF%y;>2=TB*8minnrE)ta(?|(q23dAGzcP88@{ROOIk3EH!-3A&1#Y<3OoX&aJoI??RHd; z)LF#}sC!Lge$~(u!?z)}V#7147ykSUcmb|vwXGw((HeWqlBwvUx=m$4IgQKK&OVol_K{$Xo4; zYXoiYuK`xJCHkRk^*EDpcnV4LukKqX%)fskMSuAmg_0!nH|tw}TW}y=S2|a$L>^Bn zDKvB@R~LbE3gBjbmW%%v-;Y72G}~?L0{#Vr;VUW+*6^h$+foC zA0QM#b5c!URyg~Gw6%&gZ>(WUSQOpdE|kY6N)sODkuuk2&F(wAylv4A-vuUC<^~SA z_=&ZtX>OJcINb$YsW0AKto>Hlz)^N!U4h>oN-051fHhoJV?)R0Di!G}e*$Xu!cTaZ zQd-v=K1FZI3R$bW(g7_sUh2TFVDK<_Mnkjf_y4qeNu|I}qf|RylCf0ABqysgvg8t= zPG~YG!g!veLyRw#{DkA1Cbn8MWT#>zx4zZgfqej&P=U6f^z4;fEEZx4zm=4mnOP_h z{412Ia5!v3Y#m<|n=OOtZGFflOt-aDJbRc%3@I)^a#T>UTheUbcSCrjRY=|H1rt4W z{^Zm@a36bdpLi_lT0Hn9Ef~#hV>MOq;rJK&F-qM;Hepe?5T>cn%<(X|MH|WI%i^5+ zeTYoO4?#xRG`$HrtjWaw(gYDfAFc+&EkHZhCDINq|4f|ZQE>^53hB^ADvN6JO%;>R#d^9PuQ7)Y+QxnR}YW-CAM85HvLhuS9l zd~YTRf|Qpi{oBw?VZ&oMS3_{3hiW%=u=VovE~xK={gcQUO^pOX1L#R^O29KT=Ow1I*S#Z*GPw6<|5)D~4HUl>I~)nHLjk znH6_0RO&i(-+e&NA<~=ZO90@VLO#pH!66pFd&LR$Ua0h$Qn~B_}fZWXuplR3+v;Y4yhnkpV4=XKu?<5>?mgy zdGe-mRO9$)H(AvBEJ%tySABsRQ_|g?O>Jfg$~fVFnx1x4Q$HsOUTQ*Ecw)fej-T9^ z|Ms(X!JKk=A_p}J&grt3yVLVwwVsFZ1ZhXUtz^vlKZh7#Xe})tfO23moA+Gc;qy#${~PyX@t4CV@I(sHQ0h< zmY((@5)GG>dJyShV4Q~Qo(}uywrH{PCiYmzgD?4;;8^~{Lp0Ofj(AB-)Lw^kN%x?y zV_Z~7`Ep8!p^rc*uBG1V9Q_^i{+P`%cZ_iebT;MPG5Ktp;>x0XnygRqs@H96-*hdN zY$%fZ^42>XY&&|boF#Nij$kiB$_{r)I48=jPqu0ZRq*AiTHy(dDa~f9q)&@*d}pWl z%B(Z`8q%GHZI6oO2^<+uf!-jb$I&Y1qU=^5hLh#jdXZrwXQ7SJAKg#UM>)8#lOfZhbbIW$a}OW~B>OACW@Hru6%Kr0%;&!w)@my1Y|?okQe(DG2Nw z^9C`(;j7N61WNv^ORW)smBsusjW{-9ORF$Xh&hqHBT>yCg=s9GCM%@_E!^`;E)g~QfszfrFon+{?fQJbEM;#gYN_5Ds%PV|z*;A;yXd;=E6bvm!fe;_4m zmrLNgyErkN`Q@RFWd;Z*S)VI#1)2oaVD+*`W#KNMT#4wrduyEW5l$t;0?8nF=KO^c zeaTn!IY+Im^JEpv12FpPj-!QOy4A7PYy%PqFAnx|m|Y?iYY8QdEO&2h)+2vFTL#)D z9}YcF#CJUXc)G(jm!hn-1z9+p%u`u4;zfxCfpF*mnykE^pI zIQ+^+yE~`PGmG_N#SVZSF0EQV>wz6Tn32T5_oR+-72;k?NTz_x-oVPO21(}d)jA~z-h6uw15i|Lc7i=IYA}=%&o_T>e_lJf!e=7^o zCZAzcVcwZVTKy+pjH-U|RGJ`+z!D1FG~D*>fksa6=C$G?hx;1C>8Zoo_UTkEc{gns zdCV%mNl*(4n9rdfVQJZ#u#oy7&`AWIlrSt`2qNVf^Vnb3)i4_9Umj96rR{~|{G{_- zo^|6KpX9aT{Eb}_ktIt6C3Y<>DxZ{?I*-OoW|^|e=|=D!=+yy}mVN=VXo}@B^Rz0N z7#VvDrIRfhwN%O%*Z^$Sy%`-hFto|c!-GQ}a%-l}2$DG_IOnj94tRgx4bT0>mVBYo z$T&XNhZV#G8_L?I9M(voBN~!9+VS8&y3o3DC7;jg;086loU0$L+Em^Tr201U>LL;E zFR?qvn|kY*iOn0oLYKM}mx5GmLory19DRRLTQf}`7l(`71jtY?77g|f#v^k+u;hR& zc?FqAG$p8hyE%r+E~?tB|9Y{M`0KeG;amcNLQ8Z&ON6GrS|xvC@Wthq<$wZ>RuG$W z0B?t=<~)=E3@-Q`5pqDZMu79K6+$J|w3=r$hkb!AekaL{5#Rgi`lpTR2}bqWQ%1l% z{DpG7HvCo`a z(u>4^slq@SNPpm_%qq$WZBR2#U4-l$)zgP-pA7VyFd>f+5nI@0%K;LxCJ;-_yJdQ zUG@UgSS0K?Sn3`)#{iMG0$nnv$ECp%=I`7P&w~fQ?4fZPHp^mQatVw~;|EvnFw3?i zc#ryL$W9qg5edr7bb~wfEu-_5ZsIwOf}wr_W9X$dmf|hPwX=)24g+-ht19yA31jZ! z6)@c{T9y5ez##jG)I-{ZL*4;fx5#)EjUITtjS-55(YV4G%X^0m>iQDwDy(%`qnTed zPVA1UnHdEX%Q>^}va~9PgPYwgpERx~6+ z<9R##s#1K=rtgQVn&Cqi@Ih9+464ji0E3_H$(15} z>ronjljXGC1DWM@LItQAH{@>as{Jzmbd^yv*co-`gBhxN52o8aKIW# zvo&DWOg8h$WV)~E;frSKX>ek7LAttQp)vt~{`u#SQg~gs;gFivWong$A{&x?Wh(vF z${#-o+G4J^h>JQvvFh@ZxPwKY(dTj(JLMIliC{I@v_FR*h_;c~^uC=pElWALYW?^v zPei}6EJvmoMr2%&wd{B+*4Bu7a+RD2Go|eowLH?3lExm7GKIJQ4vrVmZl^P+-Eq3^uK zAbz_HLl<%Y7A)d-Lb2>>~YF#zqW&}xhX{tsX-9zBa* z?8XnOvZ}ZY-==3Y)#8ScST@AZx~??>s7))R+N_1&oCM!^xgbR*On;kq^*3hw1P`r3 zmA8($O->K9V8dWXZ!|q~Ni*A=Ak6v?ke0Xh^F=GE`D?fJ0=x`qF-(|mxduDznC%MF9gS69!D=UgVc+ z2LDanQ;e>OM9g9CDc#I4o+xY=u^Nr(6K0!Fx4aBN+|}zW6KAfSY9Rxn77%)CZYf&k zR8{pU3GO@cv4%$1M83J1nMR`(7LWm&xVX%xn0b3=$fmP3#sK8WuZzY?X+V$ykSNiF zE}8jnm=QidE)Vk!iSs|tkR`WkYu5VXLZv0doi0@avpZk`R>c4ya9dkS!t*EWDOD3p z5wAF7)SvZNfJv~4QKTXj4H_z{P>+>1jdAD_4-XgDLoiaC+{5IMnSNjaUHE@Its?s*J z&7MElk4(v17O#+rTDT{spAduji!}5YD`1WX_}R4+9AV#zM|w{EjlKI=qWYpF@rDY0 zVhGbyQC{m+PUd&^S9!e5%*ch7(gqsNb+hrLrElE*9^7q~YXX1`c{RuXItL8+V;1}S zRs!r<@<;k7fETJ#t_pC01ZqDu5+r?KprvV?6o#_c{Wfw?ixTg02s>8M3*Lk0(#?M3 z{XlNCpRCZvJKTk%pZjRf2GhHo>Glzi{nGox;YZ#`MkC~LW~s7Hg}foY=(^Bv;rGuI zEm-?Qw*%Q)A)G`z5)AgGZolZPOtM)VE$Wi6_k5&Y5Q!MQDFMR%*8} z$}Lw$7ELk|oxhW9f7*>GWu+e*vM1 zr$FAw=~`J>T7_>TC!okeuH>^`uk{S1Q9krf2J>w1pJhcI#!*+>?u*_@9Rslx$c$QqW~1n`B`BwYGK_iI9WCHYiuFUk$V#y|GMVv7gKJ`p6H-x)-h#V}B_rTO z@dP>Jwg!M7KU({2OEFInHEMgAUN~iqg@j-i<>Ju(lR=r!7KW==iE@!x1edm6>yL%` zRgEl930@zZI6Eq_-dUC!Nw({}@Qp?5zN&)m27E&a9O-!5=rW*PWJ^R_IDUKguO0)y zl7Xo>AQrq2UDt`#mjISPbe89CUfCPa1BywFJaj$q!ok6P!o@YTvx`o}&CUfb_}}zz zyY$OI1tV^3qq0H+1Q>R&vmQDq7PQ?;v5S6lgexIhY6kx`#KXn0{6q>D$L=5~Q|y${ z)keSdG|e>5oSo)cxh8O~hlwYy#W8GF`=GMAnfpeN6CX=MXJ#w{tS6hDFK!SfCj`bv zuxX94{0Xu;h6&{#E#L}|r)zTQZYF66H79i49OHm+nsb&f{w|Ymz|mP{G8M5?s>>?6Z*-O zAnY*PFAEjWRsexb{S@C=KPvw?h`s}&2Ov=o0~4kG#WpJd5r zIN-1lo#cI3cRsriLg0IUni{?a#yvak{rX9k=nQCipblcZFfILs96g9X6-Vp zlYxZmy#Huflid8tfEvo>9~&|TfjA&vz$re?#qKd4?q`SD zXkbkTa^K&w)Z?(66%x1zJ#@#dA1yCG@f#2}qffuR8vfBuckr=^$R(DUb}mkOl1Y*# zwVRc)@NzA-4FLC#FMQ<8NAa|ax|A(N>Zzjn9IZ%KAH;mC08$cP5)Xw$&3lj_!Up!v*Y$8n-7r2m_ZrAgCULeI zA4)XBgA&7%YmrTHC~$XCuxCqzO)6H_do9IULkkCrqu!pl<48Mubu$dUF0Gv_X1b;s zQVOPHiwY=o>E*L5h!hN)$|p6ra1mP9aLKZYH1i1BYq#~Q#fe%%si50tSd!cBW+fxS z+UrYvM>tci^YIzu4r^7q?+eC4NztqWD&|Q{*HShVy@<;AXfDKO;`YF3AWc+{oYPw`yleS)S!k0h#J zGB2R-{Ef`C;${%()i~6Dd0ZErZgarIJD=#ahzlXYtM$7sl-oxBlSr_0Y-Pe0<0w{> zV^UlkKhpABJ%VCJ;+oAv!YZv#hFiB& z{a-CWzSO1HUFkEImPY7{ZyI$y&!qJ&Q*lD;jg0U#c7Ut(0`H>l1yNu8h~3=xQc~%p zy;iA>Sr$nP$+94rQoV%L2Nr*pC_c=}*@8vlDS+67d+U+a4EFcj>eGRY6CKe?YAlc~ zQUx(r`V5FjFVm8h6Vu<5otmDLXW6XwA<_&7mt8r5w6;o=J?bI5JNzYwWEjb5? za|R02)Rg^(T+q#ME6$R!bv5BO*BCo%QZ1RSAw-(Jh2!$F0Rnaete46GQ+}suWDv---jmrk85`$ zRvySinSD;C&AlmcWHHTP7@|6AaeQJs$$gHIS(Pp6-M@->itRLvwmYJuoU-zKjQQng zfn8nyB<9Xf&IKa?na2lo(*@SGS-Wq3z?e}iAMI)|C+LGSwLlAjC~XTlSJ-d@(*!`Y z;PCreoK6@U;@N){l#~2q-`nLq@afTxTDxTwWwPOSf0>#oLRM5plBvfQ>-xdxNw}XL zq%%<&!W(L&@-OXUD3i^-7O6cvFsT2%4ofox^AuDFLzGcW5;1Xw~^{^M4K<19* z{cs^DLYCYjOTy9(Yb9{Xj_af!`=V^$HIm7`$uL{n<`Ds}Aag0f9#Pu5m-$V``1?z= z{ZE17%;t8O4GB&&fwwWL$7zGVk$AP@7fq2Y%e|6B+th9~MHwXKCoYiB({}wC+FigBk=o}>=%VUQtw}cm=G-rbS%C!B}w?^#rcwZ zreNMONBQ65^+XFtZ`HC>!m5A~+itJhdB!9BBhPH!LyRUo!)1^urw!nXOPO%|?LL-x zLr4`XsXHgr>e#?bg&&|U$)s&%44~>hKqLpg(!TO|amF=?#VggN$5$jh)P!J=2#N?F zrb75Vu02r7HRmN-mG+dLFcN}dzu1YUVTc)u5yu_hq+O+9Ldvf^$-D+=epQ=$5D_hM zLvLhA4vL1&jw2)>0Hky7ffW`#<6jLuItIb#wD)%b7})a35~^|nWz*7ZsQ76xZl0O0 zbyfQxERz%XZHB&I$Emu?3SF3AD7u;{gWYQXQVLk=I--PnICU(ebez{WutyKxznpW26^vfUe<(lnoeq4P=A++q);C zI+wCaB0piFHCz(4hkmADTduGO+~#`h!$d2nwhqC$`0!kXHd^_L7>C}s6Q2Bw&e7K5 z;Z<0gc94y&t`YCjQLU8$d{Q7n(>sRS+S>dFH+IRmg5z51`agT5`v&DS8()-{?2C>o z8K&4KVIko&-%UDMYl&R?~qd zN~FGa8oRI(2Dn5`&O$`N~?9Q+TeyR>GlVfNZwNRL`=ar4* zncUqL8AE(-pNnT%Y;2o1j$72#_m1O<>(3ZN66NKqVmoUzA)e5)~H6 z8o#4KT^F27s5=bi1z)0>2ko=&&?lnvTB0bNrn!HzT+by5yawG(4K~8)%@F)FIGT|3 z>SoWLe5Y^zL?oIdno6O&elqR=!NiJ$IeC02(;hE^lr+B@PN1D+p`9|;h;!gWzw&oB zsSv_U-mhC;n#$2cxqX(jry);LX2W3BpfO#7|CvOsF}$8~g(u&h$z=~qGpw~cIP10M zd6_^ZfbU&iB>+PMMZw=laTiPH#MJU=;7&i*M%V5)L4L2Q$ubRB1LB`c^A;H<9ao#7-(QQCzVnL8_fi zn;CV1q8z4S%GO?(XjH?(UEdk?xl6ZX^Ve?(P!l?*0zndyknra~;Jy z_x#S;XYak%UMmktD2vU!EiG_O?2vJjgW9%?92rsoRgITf=iz^X?|>2Q2ykY7 z>lJ36JBob3#3LNu_!ehK;(vBc9!eEsnu_JZn`rlyzOrH!!d@%wFBHoQuL> ziymF0P-EKc_s0B+VSB0wKcL|-UviZC^Xs;iv``?wtAkP5p>?&a!ks*wg_Wqz5xxl; zK`ikgM^dGPY`IoCWyb-hv9<&*9TjBZ%W8>E7fSV2({p|;R@ zW$;`KkY#c@1^@XCtyWV(E~!#3iN+#`$tYZ)R9(D3IY#%VeYU`yI{C-v)vNENy~{3W=NQGcGj3FN$GQrG*x5M~y>ZQgo1-o3UWV+T<)y$1 zBz`AbXstpZ6Q2?XaphHSW2svtkWX+X7K!rTbMsVPJoR_k>G}WIpRyF9u(HiY-TgXp zd3238#t^sY`N(3g2a@ya_S-60iV4wW%d5+0=%l9IcYFu-ayeX~Eg~Tjp%V?gEZb6N zAjtdjLI{qqTXdXLot>Ql@N*P=3Z*&uHEmwHY0;*rC{D=<_^(yzbqoPm0}R27!38MO zH>V^CJS;FcRm|QNLhFYWYKFEIF4_7$`@sG5bTL6mN1`nEndW%WW)sa= z=XcTU2~WOr$fDq+@Zu0mXv;VBy^C+3bWh@8Q=wwsx1vOwS9MjY1GcAp9P8PPX5ciOcg{`wr!FZfzJ&<{A0Bd9(j!bkOqubD@!pvWy)6 zE-CploqV#|kqKGJ6>{1#OZzR3PEn0YwUqDiNlNqxXPxLK zqVLQI9r43LLPCBYCL|G5Ay$}%nlJcqY+(@>9lm@&P!|H5oJHjj z!~f}vWGlGj4j%-`kl8n!4kfCabs0=yO72352OM_KMfFg}i!_xolUCP$Lr)al%gjHq zLE{?yi|_@M%V8YFHbeNyJvsbW0Svn8nq&34nTmN(qNNM8#rg5-yT6u9rG8AupM9jq zsW^>=I*2nRo-AhJ@l08}ZynPlD4;cF6HrLCuRVC6-TI&~J+;$YWHOl6^_(pF^d}3~ zfD2nIM{+11nY8u$=X=*08N9uH;Dat6 zku{oh1%<2gF-)_|go;YDkDN3d7Q?Qjg6p6_j3gzV;DW`J^Hdxg&5%lNqak5MahUkk z;uyB$7{iNIsK5tZr z!};+s0bzA4-)Kb$S*MQ()%#JU^24G3xCfU#3Mb_x$;8q;a;f2@NOQZodVqHgB!~A6 zZL%p7JA422;XI%VfV>|7#X#v70w}Ds(oO7yf4chL^n)HwRB8}R?WNG`8rj%HfqzeR z;5cOZn%-1FZkhY(he^`6=qVgiO_QS35zl!;${l11>l3E)TZ{g0wpx~k5?ToZI z|7+Sr21b##iXH8picP}a_7qL8(t|EKMY*b$WevS*T49|BG~@_hH?z!|#q9wF%iG=}X`n{g>&>XnP7;ejlb{m~q)u5?KbZyKG$gg)n8}{*2M_9anC+&$3mc607MjrSC!~IHk(? z@X9oD6gDpcSxffGeD9#vDOu3e+ ziUgvEyyyyT;hVAlnwPzWmyTsMPj#bbXv|PeuEF=b`g`AD`*|5A8pfYLE?Ydu3!(2u z=SD8myuc5A_=5XWMf5dA(0ykcoza7f6GP(16W2M2fbxm9FGEG&{W$$T107930U8rzvL@P|8Ie~OY-;6 zf?ct9<Stdmh?jW~1`K>7U2fi=W3zw>MXga1oMEKUSB z{NtlE=$Y0U_iMDf(E%H8KVC#w9XrNP7(xaxNZ&xjisP%Ew0K*BH=y(7#{M-K&^peX z`EQ-yrbFLe_&gr5Mz!_P^RBvl=MnN}Lp0TlTwFr|@7}+;fGy21cE4%Qj%9N8rhPHz z@VR4r)1Bq?xpS!cX8D;$o2kS5q?pFCtKW!)yaW z4ImIn96sQ!#7kQ0KzAmLo-9#+p855GS4kM+X1lUXuT=7M?^lSliWX}{jnRKBWfbJh z#$o5KU))RPumNST`18lbS$#I29YVEfgwMr-VO-Z`as+S=C#@Lv?~^S*60xK_c9h4D zwG^3z=Op}Lk?fXq;A>C|lDCAKkqFHs>kXDShBE$bC19jU_oj&NcD#hC&AsobhsVP^ z4$6Z&OHC{xxX3=zHxf*<-(j=I_ILI>Ak;9>^`QP53B1NG`~9E#f#!AwtMy#Nbtf=a z1mbnyfuSx$w&y<_pgmUzx7TqsCGB(9OC$#O3jlIynVEkdt`C9EFVmOPWojBi#sm{L zw`8!10ay>feE>t7nNOLV%+alARkl+{1^+SLB_DUCe2>-49lpQf1Wi9txn_z9zXH(- z>%&}0JelBD-dBXjuKC3oS=>VrLFzfXqodFXVQ_BA6gH!NAwYEjNe;;)S$ep3DbY8+ zqDsEbY?H9*Di75)mlr#2CXzzIhE0>3Hz1#@YX5866hb1Eb)q@0M&*AR&HM>INrBhV z9riP|xKj*KeEzMOdFmqtk!4U`6&x?3vkd(+*(#D>?$G?ps14JvQ!O`8BP!s59oK4AifUw4d|hlF!c1;}^}^|?M!WwTj`rIF2_92j|Y8;X1{^A&iW z1BoTU9|{{0RN0KtsaW;go~6~*wuXV8-<|#v!tcV|=RK!dQoeSs9+{>kQVtVK4wti= zQY`78@U&7hoOSTOqU6x~=wHQ`v?69^<=-wrxmkFYVwv9U#zG|zGV}PST-UqT^n{t^ z36Obh+Y3m&hKHt)WQj=f=picveTZQ&L2uOGc48eUP_gDiUM&1SjsfuKH;MI zTqIcE6~DUhkD6rxz3sQs14wZsn>MCIA|fJAOiw42`zI(}Q2Ad_`P|g!>>%jhK>BZSzk#BI^8zTb*u`i5_Z)(xj^Hrn&vx+0& zts^*kZYL{?dNBpxOf@~S=Uwlpij@Qlpy|VBA2I%FMHN@1v-P437K%?1&}9_y$7HiK zeO3?U;qm)Oz!s@{#BkJ+luwU^-o`Ae`VAiUnpO)EuVcdhKpn=&$hSV#&XMN;q2|rN_^z2Fn=ZZv2JZ-qHHD zNQXtSpBX>lU|1=$=dHVeS<{KYn-M^=ubuUaa@sc6CGZr+=g>Y*Pm;{t{?R$K2ub_z zG!0LT>+Oj`8Ed|0f`AZYnZjpf*3Z)jo%PXnBj(npd>Yx%{vVK@TSnw z^}O6Q`Udp^=G1u=^(sIy!AkuY0JGoTRUk?Jwafo{zvAhyQ?~_KukzXSL<|keZ0>Ip zeTY<_K3{I;@J9`;9-%QL9vklm-ge(frax4%g?|cFA$jO@*R0ifJ**+UaBHeydp~MW z8Mc%>nMIub@WfnSs+d}2s1HYu-7J2{b;@Y!yc!#;y*c4MQk!@zRk24?z$s9%XY{d$ znv8e+iYUmQ#2kCIn$}l}!(-fgYH4U)Sm4tbpgUb5-GL~n^zM-~&YTikU?ldC(~ZLO zt16z(z(QXzYPev$5o~Zb4&JGhsol&>NuBIGM>lTIAv_5Vq22jmm)>scuABQu3n93n z9p)XUw759bSGH_%5S6N5AFA&`3LydBj{{}*7C?QH=YM92;eYW^zx7}rhQ)^xdF{aVg_yB4H9w><$Z_LUul86r z%X^u<5A~u z=Ez>LA{?iK!laAyw;NT&ig?cBZ<*?N2OB6kI@=!*k2kWHt>cn}o+h;UEBB^`h5^9d z?v<=k?HkD5vjPGj?bdz{0;@N`#9sWc76P9Cml#@D#t>5`&rK-1{7Juo2MEPydeaIb zwM5shK6Q^S1yeQsN@T5sns&U?xdK1PwsH^4=SY|qG{8No`~8k2*Fex0jq;yi(n(Sp zr_&%-2RA_*pQx+Ruk}@>W?EH)vDl$2DBs(E+;fG5isG>sXHUW945J$}IPoqric1XL21MZcD$6Ba&x7gDuc$P`2lt&&qJX?$8Y zOmCL)iu>W(vN7dt3z;spGYQGQv-z#x8Axeal34r#{D&Qhq1@?svf-^a5F61-KNF} zC8?JUQkgc`09P@$gl#mQT$C zYzvNrtj+e-iqk2PRWt6t)K?4_Od5S1kW;+)`?AdQYB_il7FvIjtPMIq_vWuhIfj~8 z>n}IW-?g!-Tr|(ylm5o_X{GjAHFjhv_}GyJ{E!q-;f}IQC-Dl-R+wtK8yc4IqO}b! zOr~QnFw@-CJQUi^a5#$G6w1yZUuk&8FD$s7g6d$mddbXcOAwdl94qzTIG`Q#DX%XN zD`N;FoQzGZp05?+Qhm0LqhZlKZ`m!rOzf&)4n{7J3yYrZYoJSgxMvHWIY}D%no1>t zUC$jMwOXY1k33d|8xAtbDIw-*8~q8Wm*5v}L4#J^ajdI!oN<5j`442@@SLZ?oWH~R zFBiTwGO?KUhW}9e9aD31PPz0+7P#q4{n<1FmEU#}~2h|8{M6+{7am zx#WJMMe0qP_cHXz|}EN@>*@QSV(q` z!i0~-e5r8PtOkQT#Zo+U4I6e(^k zb$0h8Z`hl3l%B-Hx& z&V1KQoR73W-X(i#r%>&8sPv{y9~x*}>te&^8!F^F6mJ9Nx;W zGH6Ilr}~va*_`5FC)r!x5c$KFlt8}_H3la`X0^;Yb23YkyazFnAmn`4l!G3N{MF#` zF>4q+^aq=)RA{#!ZQLOoGMn4W1vL>@?UDvg@Y zGyLlUF_!6l6664|#q0CXE-mEIyH78ceg0)n0C zoEMfiqWj%|INycwM~fG^H(?lIEZ(qETb3*qb0eECTJM|^H_9!simGUyw1@tUm!16d zx=<)xJ(#H8Ve;x}Xyn4@{Secoo)t6_x2Qn8OXo1=W!8@yIPN5uMJIW$gYGRyErUjp zK^j6sI9H*v64Lmj$yK%eUTj<_i2UaKzn6HlUW7Q!7bpJEdzpIYXj~8C18;*Zufllo zE<(s%9*N3G=;e4DZSGO?*Flz3Zx`ROC@J`0@RY#N=28Hh&U~M%Zs%DD!x=DFQ~sQE zcc9)i1iGW(j=zdw{ky^~3YyADnk)vmq|YTrP($Rx&0x~3NYNp?gZ45iv%OX&p~8Y8 zpJ!EL((`ujiae7aA*1=%QY9xdIp!8Fhxplc!2vN2R6^$&o9Gd|-Xv&-C%dE-!(63hZ&jhtSvj>w8(znjz{Rr6c5>^)NTgFX`Fr2==0Ow zMz4MquEF>k7Wa5Y`RZMl% zgCaK}5rB&w`26@!iTWfmuuc+SouqkzYP3l?OJy34kC8cATFpykl}$p@sG?R9fgi*m zLuzn^v4o&Z^vzpuZ~OnQt)z{l%Lq8?@q`Yp>^%Feb330NW?%B{y6TZ|aiu>!`N&YB z_x1Jp0FAwwxjER%90VMao`OT2y3hd$RiD)aDlr6LN<1pnA)qZQ%5Y}V=k~kD#g;}v zdUiF)Ra~cT4P}5yv(jM$AvF^k*4>hF&8G_jllTNO@s6SLQT2ZLI`ZNw1Y*G?D$FMQ zJ4gG+2DKkV&_&$9K?r}YkSe9B8GRP6hvaGgi|bA_)&@tku>+@HD~ye3uJY%^`-h%D z(IUO2^CyG7Xhhvz##Qa2s;nl@$lxY-`{bgdmQL?C5|eY;;A_Y(sI?4b9^|3LKvom*EI55PfYgKmf-xhyGF)uiR;X3)*e6%3M)mp5N+ zb0r|`zZVK;d=?P*H;9)D`}@bq^?Ilzr(;FqDyubaLqD;i*l2voJDb>jrLjDO-}a9A z?#Z&>)mt(!&px+pAcg=_1*-k468@#jXu3KA(iUQ(gN?xgvJm?5C3}*)znlpZ9Badl zrk0kz6k!QNPs2OR%10NB;`6(j4cU{){SLA-7uh?TX@c^mU(?s66pHS)T-K9Z8<(Df zp>SEhNA&u9II#a*A?y9;?&`{lfJIBp&!1CKLCgH*3-@Y~UiI#xdCm?HvwGW|Kc6m! z|L*SA169o;nY13zr%h!xjnrmtSUz6yy`|zR#2;9SoNcjh;s( z5icSr2-&x@6Amj*+reEvdR-r!!YKnu@sA3HjM6~8thy;EIgA=w?&rx8^6wrMV$IDx zEQ|kcCrYqPyI;R5H)zI1x-JUgS;9sp5GOaDSodn~bra>~yS5v#7nEbloz2QoD2QzE z)54frNK-imwdsUuHxc~gg@C;Xft5;^v(#JLB_mk&c+__Ans8~7fN}NevmNBafz~Wa zsmfj_KQg|K<+7psPHlmn#w53pu+y7}8A~{|omd19DXlrEK{IB^r%*{)8UAcvz)}9(8sGWH zkf4RW--pI&h`_n5gR#o}@+_z&Cw}fgFZ_bjHvdC&Z_~rOCwR&aE{HEEg&DRc?06nc z_{;2ntGFWExW^9_{J+8qnl)@9j>pu*6V9g`99Vt+{CS2eadL9f3dvMNBzPV9I-VRj zB2=HfJf2?OcD-f^2EpDp>WeFAXbb?<5Qp6ke`jaMip9>zh!hhO^SFtDj;;_G;~eZJ zCStq|9GPw(eh7v`f2EZXM2RIIG|u8IpB;)L6SuIlGXpx9B4T1nUjL%B1aNuUJ$2?w zE9RrToarqf%m4nYWN)!Uk7_{kUpn`Yp+|XkIBsh?L?WnV#B5Lz+N7DdwS!aMA|%IG z+hM}55)-x2kxpFVk={A{_*eQKNnBS>USyZS@T4C#>mb@C$*sYGSCNh*dNTC2;M8i; z_gnAQO>Q2!G=2LWcOweog-iW+?((VCr+k{yrkwVx7lpkk>Vd!q12l?-G;7+;((~G2 zJOVVAC9Xub_N=8PEh{T44lsxX_{zayVd!{x<&h|R*T?p&s;a8A+D%af1>|*gb#Tb1 zq>!fgsLfx@Mm9pQ^@x`!e8-~uF4T-0 z*Ou5!w;lr3yzrANN?d3V?P1a%OZoT@inx|!X##%?-eDY05wzY*|EJ2a@>^|`*_7X( zrXX>XcqDqQzRZzl#`9|hH1WL<)$T5D)4tN53Oa>Z=dkGYDs%qoM{9MG#ir}{x{eB$ zlQY=cI9w0rra-($Di3mmEIDz|vfZ1j%*fZ(ENX2{0}jk?YI^{I85kJI{qZ9Je5V}X zv@<`F$yqNQF<{lmZ2Sk&uKk#;NInONW5_QlDJd>BRYFq}=luLwbW>SL=|6fqbT@ix zY7%Pd(&0agm~f~?IS%5fV5LF4K_?^}Tc|Z=`h+-87qx099#7t*H7zw!Q0d{tmE`!} z6pLnV-#>?62O=o0vDQI8<=evt|MA}$8@7?Tss|#^C|%E(iPP!1AKy)PHSg46;cu;Ox7PiYrs3tH2c7R0m)$BqhQTu@WXPWz z)d^_n7zjZJ5*sk5F{a5}9qcTq*9x(Nai-9ZcyC(?j0Z6F#}MsK%k!J|1j7qvG|vbn zo7#0fyMgem2^zy~PeAqf?8Fo@+OW_c{Uod{CqhH`L;ZXkl2Jw(^gJ^*jFaYAo<1cb2ix$YjVnJ`^z?19D zCdYMzD|9cK=tCuX9F7h{C5bKmDe%sZtD(9)FUHLeZFD`0-B*-YtFE@CDSRDqZ+|Xn z33bnfi(Jy^bgd(bh(9|s6MtxU_#UL9i@eC;y7&YLCcR`6=K@zMQ`V)y_rh;j$^4#X!WBWP4qRPbfV zC4pQ&G2Ag!AY$c|M-{y$PJc%Wd|2%dltQj+6Gj6WxuX4KZHvxLSTz2Z)Qc?cS+{qs z?ljA8lzZaK`r;jNRcGQXt3+^>Sjkkwi_<=}`p+)OMo?9xl7L)qslVXnXV|?lf#(z6Yw#ujiX#L>Ir_6U8Pc&+W5e(`o%q zrqd3OiP;4r)es};tl!AQqamVw#ugSdZjP7d?bbVeAC1^DCih+TuDvcUR!RePIN(qz z*L3To6vhV=sP<>eQ_`QxUAwF*rRYRD2(!$_l$mrYI8Icnex-OR{zk!(Tu4%PdI-gn zXune0+TOlDxE$}Wn&SFCzoYx;F5YP~&-X43;X2d+)l57nY$=P#Ud_Zjq8{?D=z2A2 z0pr_*{?_N;n@h83)rUMK%}vnwm(jD(emYY8Xwk-w`2^*ns#xJ_v7+qjjk_(lyYBkN z=eO@SjFG05i`Q-)>30jCzSRzla4vSo6I$xgL_xpJme#oJE5Uxm{T)rf&4~L0r2KYTrH=DQBa>&&$6 zqpA{q=&K^QCmnvuS)eL0X$28Ltx~IRJBEL(!E!qAVC@a3rlzHJX_rejeL0WuKkIsZ zX0UjEPDLdJmXfI_fnHM5(w261YDpYVB+?FWHN$~lRZ(gN>#SsPE$TnZ*4F-B^B`+G ztTvyxJGbI!h4S@iNuaFka>QjO{zfVyp5L_z8~psZxgi*{Y_+cYs)iD76l<-$#0Ldm zd?w~*M0J9fMXIPzB_)W16t4`utlPH)!;_F{-CnE>2C(QGsiv_F=QdT-`WX^8@v{yi zt{ON~vvvOLq6MnoxHt}bJ+ya7s|$3SVB@HNuI*0$YGeJTvku5@d3kx^5fLP8Y{>%y z5;2}D*#>~%10-ng{wB!Zjr+g6MF6O!uKIz&K@t*@0LXW+ExHVFQ0jV~6oJHVTO*lP z(@C!1y$UP@wu+kCn=HATAg`jLA_gX=qMn|dj0_^EX~@{v=8CmVt|Bopz*LLq_%`0NMagSlBX?a-SV67Ii@W_c7zgwe|E?7bu6qC!UwUZ=>E{g(+%RJY8M5X?ZjWzzBp&& zpBpw)-_3r&Y(cAL_yOerjWFA;i47N#^}AdHQb4(sOsGlt`27cc=NvhnQcS`#(={)o z6UDi2;V#x#EiWnFKV`SFkC&RT6q#c z7>#Q+R$#9|3d`(vq;cfN&C1HEqN=K>q!id^^j7YJ*kWXCygaD!vE{`JR>z+gf>l&h z#pm*4VUDjDh}zS(D;!%@KjTS^`_V8lcN}e&t}tM#2sw*)^1^?Z%eSq2(MXo7{jirM zRa@yj*%&l9K;-na?s{kCM7UbND}%xrTcg{Kbgw`wBuckXId<3}Pw+CPJGWh_k^5asOj6cAJ4?pvz_o+a4;hKmTP{YfXR=l%YCz76>zgyBV3OJsRyM7TCm=h1V6?^Uiz! z3Mr!R-j@{(uTuCyNX`2=idJTJBO-amzWVAPdU|@m#iO3&KrX3NIYFo0bT;y39HfEZ zJ8lCsf8%19f#3eZF~{cu9tQ`fprD{dcYR{wE4d^P2?Ig+4v1%itNoahaHmx?oY4_w zv_vsK)^yQWu8=A`ucS>K-1b4V!z?T;3c9+oii)Vf9R1UGF;}GRnNnrA!0v%Kvgc^0 z=?t0NX%cv-(YmOl8_x_cMe@%SAq}3FpBzp*_Et429^@>fhXTK%;7PBJ`-NU#j{o?{ zW6(2WM=+u-ARfHZQ3>c@R}%mD7QE<`s)hBh1FDNLaG9RBLG0$NH zQNkl5Nk4y<@@2NRvf6s9xJpXkI`m>DEec9XBwV*YPu66kfNf=|t}m;pM>tw6Z)|HJ zP^W=H0WX*g&z>gHvh{)9?@Z7u3TL~PK-i=}Jf(uWgdH&-m5D4Ua({ZC$r>x}i`j&d z;^rmcaaCtgpum*W%5{FU?1|1sOg|r#W7h`(4UC(H!pkEI00a zXX!weU^v|K;|k6An-hmir1NnGtPdb_zUj)f{7bh+ffmFhJ|o9Kq5TQEHFhaoz&zzR z*)jxfYHDhs%imue>G*vZs1#DwHEIkX^TlJ#rV1nx33)g+EZF2NEG#z8HhbSfoacqf zl^3sCxoTGbf3&nB@#J~=`Ks(y)=7}gPR#0E7a!_qm30*afS)?X?mwj_b7UoDbYX1T zjjf&G5J-X3jjF42xj$a>g=XN-Fg2FTP%d z>-j_Zk+S4&)!j75l(EjTgX7_ncJ4mLr=OZ>zg(C+Nac}F%yz?=+yi)RxoKX*w0rlJ z)BmtKs-qdgWs8-Mnr**{1pr+C%>u@5H79_lr5&Bb<7!g;9<=X8#R59ST@jXMuitg;vjH429%+euN7Qx@mZCQ5*v4yn(3!onKwSeBn`v%9Y~k3~ zYwj;?OI*M|H4EpElX`nto~ExC)QAI~Ge zg^SFl2QI;VeW;waxJBA_n|Zay%cG56^^B{(N~r|D4oIX^%5*#Z zg>#aOIK;}ny{^$acX2bZn>yyw=MLv8u%limS_|`=LSr6FH1^oyyYg%`XynlLhr*?% z;$M1?A0TY@`9N&C<3FO>zJK&J(~$D&HExew_;Hn;;688^gpUsYq7PkJntJkA5UELZ z|9EwoPfyK?%+NJHyG6O9Rfs=<@Eo21uNZHjZUmUe3|0F-TfgxxS!0+!oly#HeDuCK zBH`uLOp&kKT0&M&9hG@+q&kP@;x%IZC41P2q_D%YRnLaPG=4l8v) z*L=*3I_?h!r81--iiC`e$y|jds3Q?HlS@lW4S|1@iMe?__#Z$uQL9x6dNbc6(5YY> zv9gHaK=?$%!I5-wVuqljqf=2=S5{Q~1v=^=G&c$u8yl~-#exo@!G>F(Fo3QE(Cc zl6&c1&y@3Dts>l}5kciFqMFkzuhr*5mb4eS+lEE-_k)*9A(3{2p6@2?E4%8?VEAgo zsJ!M&I*ZLe(EZOW$~!yA?AtWkL5CiRnZCszMJ1V18>7p*N~!^l0VO>}p%{Zx>TWzXmQ4{_c1HcRWLCW%7A;NO&x z5|Pw5&ae*%rh+urI|%NpuGld=7jS{ZX9Hw0HCD5q^}LT+i)~k6u(GOMl8Kmwf)^qG z`|GmmI$tWP4SIqt*K^vOK&b?1MIZxJIwQahmC59Ydy4@M8{N4uXpkV`K|@26kZ@>; ztWYcGLiXDF4S(d;u1p*!w2>z#9y222Wn$)}NOMp0dO#ByO=J!#q?45JI{L3y7{3gN zoi+rNPFut7JTEipg|bpw6=2r>;hI8Zg_oemRk}6X?9z9Qt$KY$<4$D#1(Dkv(S&AV zxVaD()$cX$Wy3V6!Dj#KW{7CQw;uc3YNQ-=7Sa_dT%|t(6rA4y+)!S0(SQlkZD+_5 zpE|nz7t*deV_p;e zKP~{Etkt_6YrWaL_`O`_xc?IL4rR9c%6-xjbf_Jo(jXxz`J<&p<5@>Z$!V-&od_K) zyd*xiTNH(Kvcs&ZIwWHU>!*IR`dar>JrIKLkL~^682fwcUHot1{o?N+A|4xUiF_(- zPyivXS4l9HWV}>v7&6gYE4ZZOzuE72VBb4Ce+fe(=)u4lG;Lnly7fTht1!nAK$#F1 zjpH$f-ACq=5Lxd$ZbIiKzTWvOgn*FG#G0g~%z@8^$o4uyC10RP(MyQnV+Lu3k1bvx zm%IK|pLuP;B>(^rCK14gZzL#$pxM#9F&Vn>uJqDPGkfjTY*bRGV? zbn~k)5ntQw`T9~L%}6*prr*h<1{&jOetq~ZhjX&7)}NIrLA{sB;~5X&08q<-C{6R$ ziv_BZ_`F_;fcU0?DaM)d9}Nvzt*h5g5Hg>^KrG2|6ArlX`lAWny8Tp~oN4asz9}Gu zebhm|2EIauZfGgGZfVOqULDy?#(x;iMm~XVMhLm&+w}&<6M&#w^%+1Z(TIq~f!I?T zqahUdO!zW9b&NhY>_#*R6$J$WfbFqmeExXrdo5U@GRe+ZSO+WIqF;OagjGn0a8 zxONU{WE&Jz&|pcnbT92cVG@Xr)}Vp^uP%?dAn~8lpKSyf4v9JBej!`p-8c8V&d2&D3>NIA>in^jAN&v0>281Sd&;pK#jNAfE zzPA)>B!@pof}eDb8_Jy&T2oUK%+w$YqU!aSC>hQEa?9{-~FnY9VFeWk?JCwmG+`bfm^-5eQ0D&QA6VbzYQP z0%+@cXKhlXxp5m_l%4KGx5g5fCJ5EaV?0)0u?D|rn@67_lxTaXd{O7BD-=vfSo=q- z6q?=P49v;~%pZ&L?Oiu3soi#gMR0LgL2;D7L=Y&`ZFntsI743kJx0LY@-MAmn`oMW zOoZoV!T9G{4}y+kA0}wH1F5oZFa@E};$Wyi6fL}m*3xkvN;`514X#*OS<|(jo`6?? zteO1A!fkDB1#)KzfZYw7Fn0`O+TKEQGyUR!ZSMrO@Bw9}a{|`;35LG0_gleZeq^ZWjN{zKhj?7977`Fx=&P zD6pL;^WU?tx}d+kAV8h}hSY<+;}R6QN~)@T;P+;ICP2IPen+YjbQb`Y?*cGbYds4M zjZ|{8`m!a5zGH31GEG~1adFQa+LoFniTyC+MLtg=eH;1)q)|Kaq~eb zs&0_dM5z%MQ{cBEzev#+G|o@qg*tr);rr=h={pWCUrgWP6@eD(Aw^-{(Dhe;xI$Bd z>ANpU-6Z;=MKF+@WmGrAC%-v7VR-d?&{j(m3|z@Ph_?p@T|)AXs$JimLTR084^?k{ zlH;oLQ&bcuHF=g)c5GH+Q&MM&l-OtTB8Q&e(NIzOz18=v6;!ewH;d*$N&-t+PU9e& z4*0)Z5`osr+htVU^}+?}f@9FhZ?4AuS}=b&I6QoK*+0F#S?*~F2d{EE;i7ui4?>UUdkaW z$6S>jC&s^K%b}`-@XYd%6Dl zlCu7M;{Fpf#|gJ_xuD#KcmWe}2hezhs^>2!gptpI4BRjO+^h2Nr1sF0M5(lSN8TSs-N|I=5?>|*`l^G@ju_37MIp3eI!SN3D^@1 z3eVunnrvP_xz4-YS%P6dQ4$mgkN(4Bj>Jpo1*hx7gPOs1KeK&}2bDWNoyZou&Hftr zd$HLY@hX<@9&WRey=IoZG1*c%Y;WM$MLfAOYoN&DNPVQ=Yg-s)AMR}G*&c7pz5iEU zEqx_BQ!Sac7;qqFDij|n``TOLiY= zEr9&-HU@!#f$=uisd~b!;ZVHdypyC}<{tw<%G-5+{u1BN4M6aELKWs_$dh;=00WEf z2|d079vIkyo_jySKJokU05tN3dLR40h9pglq*!plsiNP4lMDja#S|D+zlyk7zyuP;}iJ{(EW37lB!ESZMc)uV%xP*hCR zb>n0mCM34~EfIr^bo2aJ%YRMY6e$JG9bRck9k@}_LmHVo-M)#^LC9H)oJ=uJ`mr7A zw;EO=IU?R1H#BlXT+Ioba^CmpwbO;-A@?cjn7)b(0_iX)Mp7@-tk?01?J}L1Yj}KpTz@qKsLLDuN+z3!vpeAZdoXec4f4TdyY|uJ2iif#IR_R5hSq0HRG<;0FMZ3sNY|f4Gz~O$O8? zDJiKMtxC{b1_;vIKs-p^C;1BQbx3d#`9ybMvXGp`V1?DIDbGXkR>bBaBSqxjlZc2Z zaogcHx~~LG6eHi?-@hGRyRE?8yu&`WxwFG=4v zLAN-K*ZT&5Itgz(57s{LR7|*_{{XK56P$ zJhj(6PmTW#MJ@BzuEcD@(f?6Ss(0Y(A$jxjjVYqngu)DNn?F*5iC6qu8FDK1OUk+W zCnk#Y@5)8dOXT&FQ_{$eP^vIHPDSIiCT!ZE#S{$fz5tkyZwe2?hrrZsN z?|kWOy5MVufVa~fWa4c9-0rXcFF&yo91`OsL8x3hYL7VDp?8;?KPI)nmVpc*lsTrIQtZos5 zhHMk6L-E_nyy<*)-!JD*#kj2S2|D3C*-&I%RW{>o`_)XKf1XO8?cj*ghLa$EecDvU z32TYtioZ-)#A+T#om@qX7LPWUCHC5@Z0VdU}Ix*)9?G3l@$Qg zNEGZs(hW- zUdXGlRX5S%(~;HO3PVi>#~q3FKqd8jL5@nd^O<@ zj_!=vb4`4Fl0RsA_$xD*PM8&o_0{AED|T08?$hQ+(kX1qs89|+&ic8@f`YX27c7dd zM>|i9{dTnm@8PK$7(DXy^!(PmAa$U|ES;R*2)(>25c!lriDz7!3?2Emn=I*nnyRNgB6O=KW-?)=sJsz*y=7PoFmC0J>R8Kr#D44U}Jo7@8b%<-&IYL(1A7$oTk%&v5sIjGwey3HU!cssOn>av4^zn_{3D1U3Fx5JT ziaG#?WaW%+_6fC=x$1KoK}Q5_lb5O!_kw86oaEZ@6-tOQ*kw`{~Z|=R5d7N;A#A_P&$kNjD>G05i zbo<_K`lHj$f(K4Gy|!9mY6g$>^{nw&cf1iE_wabiqIOFGKVA9Na#2Z%!(+2nX=zn{ zKjB0X9~2i&o~iCreXE>zN>kcV{4UQ%$t@ydL|hAioYpYf7hgP zC{r?-VCkHo7g7Bvv2cV%W#Q=~oS-KzdP`_dIUH7Zz7!udOz%zasJd(U+v@;<5kV^m z$SS~;(h|jRHQ^at@c``v^Rtt~n`UQVZ(s#B8nzFjfQCAMEDRO2A+~hhfoEN{Cz=UH zACtk=Gdb_ycYZOem>MaCua1Q$(Ed8;yMBy-1X!Bw}J;_ zb_0w;c(-paHx|$*Cvs9-C`3j!`3}MWmn}32-Hi9JCPOo}q_Ajyo>B!Bn-o&=nro8! zzM7XDRatA3)9l!abcuJuyx`+TXCEu}ZaqESwY9Qonc}?Q&*pu_UA|ec)cZHsaW9-f z^cb_f{l-RD@A16iPV-2KXH<}AuyQDoU0S5Z^(k1&2$w z^m!*wLoI{Z%MCFxdJqqhfoQrg>~Na%b2Ur{+ZN1Xy8kH-!{oO^-f%SdG5P*eI@#=4 z}WqUPTQb?zokem1D?@IfKy;(w zi285E<7-FBl-jJqN^;TaJDdLfeSNbDajZ7KUK1M<5@)~jr=#|mQrWugI+5YHHuIhD z_gJv~mwvV?R4xUs-TUGzl`TQ94Ib52X%%kE)hoZq61K$kz}Q9Hc@vQHgzr*Kx8 zo7B_?>j56k0K0U{EiZz?-gLCwYDpU5r49X13W|z3fVn2`zdQa(c7X-Eb8cf|sFAzI zodpX$RsbdtyK;X~;Y7z_6mE8ccA23UZW8}b zV-qe0$ybjPV}q0P-)#Zn2314DA`Iwz)#3xB{*XZO0{}bV@bv+@;s4_K{;Nqdy1sJ3 zl#ed`W(CNd+!=SE;|X8b-@gi_Hwo=sw&}D7y{6&=Yq6Q>rL~cBFE3}re0y$k78jvi zBF!XzoET0N^~+zZ(gYL6i6#t~CS;?)4;p&TireayNI@y&x0IgKl`0oMsqEz?Bp1yn zLnXt1Lk#3piD~q!+uIDn^@4(hEkxI^h2H*jr|#LT77q41=VpeijK)RpQKO4gWIdSI z6qm|1b*B_kdGzpwCgi=g*phd-F4kSZ%jZ6TE{qVhoS5@$kjBlptY35OfxROmZRf}y zifd?4!l|(xt#J(w3c~N`=m0h;r>Lmwx0}rJ+4gq{&gE9J5&QcDFzo=8d-BT4)?LYW zYESpZ--XW`;p-)8p&LK|ySF}?t#G6SO)%h@`rzsrZm$kl_VV(((8EBZ(TxsYG(ZpH z>RMsj3?1ukju%M_3(g>^xbfqM14n))CBC|BlqJ>?Beg}XgU=^wV8jr@gh{@adY^@# zUx-jWkzC)XL0CM&3i}<~^XiJk&Rlf7u(NqhnJp6v4kkIH7ykFlCiyRBoHW7%Pt9lC5xZ=5~Bpkx&A1mP)cAIsa(4QO)rCj*H_pn=Ps) zo6^bT#hcf4u$oz>)WaXNFRME5zogAqlF#mcB#63WrGr_?AcTn6R zG|pGorMQ%o(S@KD<)G=@R0s3Lt`v~(#lCsd;j>vMXK0ufPl$^oE?gEY$qe>|b{~&` zKoV?{mn5`V`k8?Qq(sXEo`9#Kn8cLfzLV4N*+C-DTiTHn@;OzwyNsG%!U5H@=9eCB zlT=VBX~C)Jr@F!GSgr+>=5CN;0V1K&>JMO<=@!1X>Mn|FN9Rle2aap9e!7()~xQ zwBf8E=$iozk;v3{;g70pr&IzZ4RILYLqOSLUZz{MoE0Jbehj|SL^ps_4JWUIi{WXF z)A6&Yv(wFy)m4<$s2Tcm^|Rlkq@mA?ztX;QG5db!&TDuiEX#qa-&OY&5D@m!TooA=V#l0P|gQ;yFUUM3Pu2KR@peKM^QZUM6)3uCi0@Ex++F$Qb1t0Q0h%;Z z6JPllA1>P%Q1a%~l8EOtihnWtzWjGNJDJb51eP!!nCV{=UzZgWLSw8#LQ!;BDwyj| zJ`)v}t}IhFNT?h0iomw-UcvF8w|8X9{QjMTb9#x0ZsPzXG)>4zv{JM00b`gu1| zZx9m^d7k`j{R$PzfE5Hpq(1~f6PMTTXeW~>D=8tN6f~I8yLY4r30Y;iwE%%%FGr_3}Dc*w6-n>IV%Uc0b0Uux$LQ#s$gQ(A-EvIA|rQht_+idq`TT< zv!wcv4W0G*b0TPMU@DU$@RG5yv4!VXxf!zLtgnmtpLyALcNb=2Q%-#8Zrv=s&u2Hr;es49q{jttoQWH=O<=S8hpqlEXKbXU>UhMKYuIC@R&rzn?PqEtE40f z$Uvrau79c2*?J*G|+Xh8Cj@+20i%a=UOi43)k|7o(?A4Ye51y@qaZgM-*(elc zcGjl$zgDb?L&gp=x&Y=FIQ`IgMe0m&HCh6q^*uXWLi7okQ*SOBVnpNx+^upPrZGUf zuf%bxzO7tQO6nYx{8ThF>Lw;Cz%Of9(KJ8Jr`C+IO39++IB?i^;I5w~sy@`SF^HJ^ z*r2TR1}DQ?*FGk3 z2M2EW5Nwc_qeYKG$wUvtD=no686~+#neMEvt$kfD@6@fZCWRjtoBDpJHVxuv(9QI` z`%QUJ8HWMgWER*q&Az`Z2p~Fz31wUI3hXiR1zlOgYw7sZZ0K3wYm*hP6Odf#VF}K` z-ZroibjGu@VpoD-!p~>wXy2r-6}-?z>y{NN#2(z(aLmx_@FO1Y(bL=iQU#({i?~?% zaK$=mDE_Q+6KeCl?B(?pTzItE1jNkh-}6$Xl5uRmJm*Mv`7PUGl)umtg2SW<1eu`0 z|JuI=LOn?B5+G!Rd|hPHAevvm0**v}xa&x4snzEND_@8gOQUV23y2?GfsQY8S?Jgn zNh79!CJIl(NgSX67gtw+1AqWPpEsZR=*WvMkRr?n?)CyBR%k?nr9e3j4pkIdT0%48 z8enPDfLOM$>MT}!B_&;3TbqJH$$_;3umrM}k82}sr&g14!GBPlm; zCIa0Ra`VYguh_?s%GK2=E|4+W>wH!w{}Bx(7AWi{AUlE3fXl{je{DD#HY7o@@oyD+ z7Hq95bV(3vDpX&WjXh_4$(A_IK|2_@TIAXu;AM@tSMq-EgXIaj0dRgLG&JI&FHG1t zqOV{2lonym9v2*hc81|#@<_a5p#+<^+a*DcbG3>g9)jrZQUuJ^3%vT z8M7_Sev3~?nAc8|c*F}Q52*#O<}YV85s zp<1<&XdnpVD^){9czB{K?S=~_FR-}L0hD;R25tjd5pO=wUmMY!KadLH>P$#V0`(&_ zOrPbkohIb}0+c~O#<=d`mZzQ~Qzl+nU8umU#yin=yRy=A5Vy4ymSTBxp$uhdfu?B*(^BDQQm0MK_}#2lEx+Rn1=_;Uk{mIoi$4Ngu~{wF40 z3)DCpwcE-tG3^^%zIFi^yX)~`ARLTBP@vBp{pG5r;zDE4+1JRbC=;UZ3RI>8uRyWu#mxmOY%u^2eDTqgBt3!O>* zA5FxNmxm1jVzHCMpZ=F@O#rYC|2I~#ge-OTli;JN9Dizqz zwQu*F9`A3%u&!+oOyOHObTTC14RnC7FRz3BY#`~AL2=sIp25w%-`RWuN&Uxry)Ju2 zQYhoNo|WAu#pG`bKgCp{>-_?SG5__t`vcA31#QB1n2%ebHb(b_QR7u z-vMisK_W((5#Y%5If7QjexgcE$?Grs%4J31?DOOYVS)H#@&Rd!UemJDll?qiAp9(H z>-4z&-==@q>eSmCc0>4n>!8X+tm4p%M21FKY^mATTtoC2))lnr$|1%dIe2(}KB?pC zL}G#cWc6$4jUcg7Hj18h$2rK!m|XqnB3uSnZbb~vfk-<3@S>t32!Bir2{?&Jrhj-a z9XR3K%7Dmi*4ORtJ501e-Uy41ehlZSFvp}6EeXWRE_tu=_0*~YCfVpgz&VP@dt}LqJ@HLyUb_bVWWDD`#*+e zi;FWMfic0fl6ilSacM&aj_-!oz(AA>$YP~GZyRQg?)sAbmJ5VtDjbD*%V3t%zH1OpzGE?oL@Pryw}w?{aGxFy=yuSyQH-nw;bKJK?UWI&ic z+uq&H{=aQK1QAp#_Wfkg|4$!b5o{Q6V6TO<)_L!)x>?cafCZ==#uS1m*ArYsCvQxC zb3wjD{Jkjr7f^qdTeef=?Lzq_0-|RGKj0wz00>myK>YJK9FY`!O!yU2^Gxfd|5vCm z&9$;WX7@3j5-pb98wyH$Sf~i@>d>kxL3hSaAiFQ_%0k)5O|;c`Y%W6SW&uUkg!fwU zl47VS=qlOR*`eNVgrEf5b-M2nkJfO_y}Od{8!>c*ddVz7PA?kG%VLs}@~YvMLt+%=Xo{PeB3FTc|MMGCNB_hBMpyn7d#AD5s2llE>d1k+cZnR z43mvh1)W)e*w``X`}qESF#LWB1_mX4{WnvIAGkVgnbrPVUjowLk7={KYsN>1NLs*c z!~q9;Ddlm-3Ppj0BgpN=B6re;=1e(>UpKow6{i5=2FF8=E`R-ic^8p)$6+92+aFKGZ3g1H) zXi>>ZOXGv$!rSI-3o8YYgOJ3;|CAqenMb`(ApZ~Dg$cVrR7ko$!XSkTXf2>vPiH%o zjb`koRC(A4)(&Vr%k~tDLSB#Hsc(ZuAQp}uC>c55`kx7bEEJrm8U599EI~ctsy!-% zFTlViG1!#AhBo3R#FD)Ug_mCj8xf={OGwd3Yyf8YZ(5S~QJFXjA2xI|7)4TqTyx`d zroiv#2*4R`m;`||D`9{fx*=*BH24!kFFRUXg}=jxu8r! z9+3N*HuNR_{f#g=2|(P&@W4QQ{S%~nKaio>a5J3n&V#n@?ghlt4MQ~Mp+WI9DI9U2 zu01$YG%+;n`ia@Q2i9-ehn~F9m^ZI^3KIpu@MqQ(N*;OTPCzS@f9?0vV5gaDusIuV z0~7{}>SaimCvd~-0{kMROfs(XzVrA8J)G%@4Gy$=kH!_eW{|=lssosWk260%UkdH* zGa#KahMiAG{=o5urvKuz4V~B>4OZu9MiB+r!*pS5N`0W)-$q76!g5A``Q`ng)378w z84Fkzs5^^+d^P*APwfRKI-DMY_+&^N_`@{iyZwspDR79G=OX~&I zh!=P$LSMbI0(0FeM!!74iWVJ*!kq#jA*qLU#6M90sOQaDkct#_(;obG*e@n*PQBGRQ$q zvWihfnB@Y3?srp?gbcE|0fS-+T{id|2nG+_Qov=<^iLLczX>Y} zA1w^ZXos&0G|+tXuB$hye>>^mxHR;e2ocasJR5(i%S8hvDdJM9`PA@MSXzw@R==xD zF_GJ7m2KtAlm6>K)&;^f6585Z03TgJ+MM3+^#}Omy^YDvD(AVe;$ujXFa%fdzqLR{ zBQ90drJ1~7l-bK^8YbUAxDbwT5EdafSN66%bXp5-y@&$Dr-u5JHXHtl!uCH8XZ3wm zRWdRI6}VTIQLCgfII-})a3-#IjsJcIHNgpY+&Y1+td|EXko$F zWZ&KJv?t}1nx?J2hlPSG-kY4j`&5*#Bx^T z+_n$5A6NG8K#mj?6hvLQLI~>lYaoEd6;`G zSY1!=qPT%Us_>hl+}xKijdgBi<>EJJUV{yJ9?~v80RdtMfL5`bswYzAQB2^T#XoZ=0E! z!RNzGKZkeo)-B{0!xc=cpP{Sh0@GhXD8k9Z)7sn|ytnrZo)|nN(7N99J96VTsaJ#y z2@_wyNMu5F?wf%oI7?S|DZ;!$^M^>>C=& zH71;RE{F| z;GJVq7}QBoaCvaSG2XdzXaxmUcw&HsaCq%Ucwu|vgaT*>@du)?s;#T*UEBTiD4 z&CO_d4*)+va5E}g)S&O_d7$!v5%F8JL{I*+Wm6gG|T?SwxsiZ^-Fb+|8_w}hy zOiWby9^Qq;0^Ab}ZT)g+TZ5;>YkTpPmi;+BFo;196w+)TunC~FpJy4D9HB)gXARf$ z1nP6q?ABW(!TCU@B$pU|#e$728AReeDZIPhB0WM|>i_{jy92`q0|Nuk&9z2f5UzRp z9g9UWg0iym!NGwlD98;Au0q&*h&f4y4*|U7JjPIwdt1EA6%FM{f zXj7?=K%!!UFZK+HM(r>!Ku|C_Gc%KsL*PcZq`p3#goFghrqzv&9RfL|* zFD{M)J!VfYFLD&>d9%40Z0Cu3zf|g5+Nx@563|B!aG1yhR0}-<4Q*{%Cnw&svop@{ zfG=MJpx&N?GYxzwgoSoMPt|ps;&)sZ6=5T$01qmhekr0E3qboRL*gW27U1Lzb)IXP zSzc~~wiqCUO5y9!v4?3&a6fN_b$3=}rMtKeL(pmLEycgBC;38oc5hQtQ>$yC+ZzbS z6be@;6aD{eZL!|FM@dFT#whA5Z*HD7c#ndgj@BEM$iV>kCq1mOu;kZzzwVS+FgD9#t=dh6pEjnrHAP0=>`O7hm?;XkIM}LYcr5lteKxsen)yXLNLw%do1a zh5X{3Kk{hpQ2&(qc}s}%Fp|jA+dD2cwlPa40?ec!xxRdGd<;sP6^B<6k|z;6y7h5E zH-b=-y74~Al7uYMCa3m#VQyN4EJ3I?>Rgm0VehRt-e8nusBz4Her_j#O%eVt>oS`e k5&T=nd{F~_b~*STj*tAU$o(4YbsYGEmQj{2ku(kXAErTs$N&HU literal 0 HcmV?d00001 diff --git a/_images/memoryless_3_0.png b/_images/memoryless_3_0.png new file mode 100644 index 0000000000000000000000000000000000000000..4971c4e2ea3ec9dbef40f1dcfb78534772a8fc73 GIT binary patch literal 14862 zcmaL81z1~A(=Lj&xKmtS4>9Mi(E8ghY>|s_^=~U-tgI ze-O#}{nhb5TTwCwOa_K!5IttpZ5tY)!ZA|eOFNi;g+rV!%-Q+e*2uB0Zt}tr1~Y`! z>B7+KXv03E=l^DSxtz=t^$H{0crA*FX>D%wFlpKx19O_GMY0BRd3yZYQlv)k)E~bu z(01`Ob5z+LMkt4|-0oz9P9H_kRl_uf%@CFMRe?8(AVh-*P9S$Bu6hRCu!%B+_l^7h z=6oCz6VoP!A&Nl8vM-SaYPg$Gzffb9uc@L!&&=Gj#|Ly`%Lc`3xXcB+0G~%y^U35e zWLaZPF&OT$)r13}6DUg+VKjcGj`wzDm&(YG&x?@S{~F3}?6k>Y96H$9(ev^q&CD3q z$P=0n5E0248&ijehw~hcTjcA3ro)TF-n&!+)rp;mj~fUWB15I;BJl z(6+_F(J?kLF^%ak60EJDtn7O;o52|y!k0X|aw)Ybq1V`2q3sZ@&7sJis;#ZPva%da zV8Wof;Oo79IFhQch>i$j96vNvK%HQX#$(D3W2`Q)#k{D2TOYkyA_+R`%7*8U6inYocf)Ao`S*U61P z*Are>>MwFd;d;QX^(}tLNB^$J)Oz{OO{|st+tzj(kIBIO<>F|zxFxSWA^ymhSHlgxK8+@Z zhLkn7qc-D3>aF(|v#mEfWyW4(ip2|gg0^@%Iy$y{ms6HWIrSz+C#R>eNsmQzD!?X~ zt#{t#E4n>sNosDEz@w8H`tx00GjKH3Phz}W`?DHP6B?AHVIhE%Q}^ciJi~~V6nALOopw~O_TxxZYW|Go<=)J<2E7}vLoo}=`jvE?un-q{@mob z=`*FIr;>lEd@CDOin99rCns27Kyq{_VmPTgLRpF`uM$VEyj7Y(S~{oXUGay8S>!1| z1Qeq0T09l$$jFpk4n#*CHrChq?f!i8X{dm~))xW~t%tLO?`{_#S-80q6yr#}C2~h< zXIo`}HGrz6p8!h)*p{5+lZdfkk6$SNE@LT>Ez>&6^_IJC8GbZut`JL`0+Rm95`60H zIhrfYW#DnM#WvrRpC=P6fsRML1PBm^Rw^TvQ}1NPu_7lR^x^Kh_3mU4ht^5z$Kz2q zzGpYCX!Z_fjm>Z-yu&26xWEUZqo=p(CI;A<;^yY8B-w`ybXvb}gPXI2O%%JX12xM_ z0T}^|QGM3LKbUJcHrHCN}8YLwq#mqOgwY7rfTPAfu*B&;*S&B9`Ebi{^ce`ef zFX`#SigN=#tMS6YV+G2d=aYJZK0lB5dzRgjQ&YWnp1T0|?jm5Ttg_dbNUMEGtlZo% zwW@5GNf;oOw^~}S-o7Q=%<-iqX4SCjji)g7nYQNA{}YQ{qoM6Q2_6!0o)O}Hhv_rc zjJ_0Cyi{yYF!rD~DPoWNm|@A~Iy)DlNwYjIARs_49Z)-Mk?|yMGE=|sKDyAkJ+0=S zuy%Rq1Rl&ALUXeIc_B{-K_RN6%(EAig3ef_`G`yc?fe2eNYcc>CKq+d)?~ddPgXe8 zynv$%Tl4L2l+#(>H82z=jY+@5SSDi{yZw4k_u5W~X?XXBKSIvpc|< z?v}b3%i}=HxSyIx&$>5THvKHqUfHvd*fwtZ$zhBGI!R`b*YU0YPfK24qUeo>mdI4g zfOH?RT!s=6T=oZhOzZvu`>YW5A-=q={Eeh`8(~);Fc$iBUbxSzlb!?VgDK8$Ua};I z-iPkd1g6Zf3v^bnm}C{p4K`l~p2b47YbE{kiebLg%*_5m)vp@&3JMBGzlb%FXtTXb z;S{PvcV8%qB)*rZ?mns^ZW16;o_Azqj`s~E`=eUSL5V|rlQOO&e3Pxgj}Q0W8XAd+ znRA0;tB)nQMQpelBu(v3<4!{_w0w|0Ta}tEn+kN3{4{INwdnqs!+!y9>F4WdJ6J02 z^~iuU>(qg^CW)I%aqTs}|2uA*fui5!!91AGD|_Bj zY?A}nh+U8Nh^=KMUJ6Z&xNwF|%5Kh~$2!{?0%>4k&(6=!e;U+$RHnVQ$*0pCmH?hT z-5%H9tf|4s+9-whqh<(d>N69n7w`1bvGo>m=>^|(qSFO+fZ5+|Ej~ixg#LFEn!)fL zqEC>6X`d8cexmzu@7^dDIj?t`c@|hp=l{%PVNR?42E}VOKY0gdUAR_l{pxe3#s~<} z(Pb{2B=h^WwgulzPE1s~%u4{4$7e@cb3}q!@XdAgWhkCmJKiPD_QR)C8vn!MUXsX5 z+K4dv(RyYnkWEGkMNt%Q|14f)v7bzTDQvqYs&r*6S*23oF`@c|{{*vp=PewDUX#6ug(w>sGt!3QxX z@(Ygd{GwhQ4rBiFT0aDkQpLbL-rMePzO|P#Z`yw8+j#H4^|rueU|;~Jrm9w7zBY6m z7#P67#N3_;`J1#rX1`Azx**?8Tgw_iuE0#;eM2F8B@&{3PXLqBoSVjmt_cYwK2VVUA{5(;U^7hV5H^{5QaSZmk?t+a6LtpNq2oS5F{87{uqdVHZ z%=Pq(`X&sq!aANYLs2AG)J-30Dm%LjX+44uzN#J?`^~D$W1tF_?q=XMz0qD)CTF$_(nMOOlmWhc$+`5 zh*DJEQk^=Oj}PtR5Eww{Np9ONxrhL6B>~r_gK5$VVK-&T+lS{@>Y^V=xM|d*x2!O* zghdkz{g3)1wRF~+>~W$ATo~l(uPAF=e~GAN*>XWI6A02+fo(kjmRBqnCb|#9RMJyp zU&OHRPuU7q{o)xQ$jKA{p&{Dn_@1DYuE`K(4cJEJ;($4Q`msoU3RT5&e$^&|u4-U| zq!g48h#+-+7*70^?3P=Evs**|dtSw(`o_w*_MpQCoT=?I%r-)jPDMlPH z4bIP(uj6A;Gjr6tsKwk#2CAQ?GWHy3@vUes14qgSJpHAX+aj?FECDA-V~SlMBaw_b zYS?)Ze8z`C$M!Tq5D}c0ZNZTX7NA%GcGuy3x~BScZ5q9kkYFr8!v|Y6!x@|o!q0e` z_lu{#{qp$d!W3vfoTGx_^$f`Kym`c$BSv#Y$?n3N5>{xmg1swH- zj^BP?h%@*^`-%uYDU2U%zD_GTZ1`{6sK?mUWK_*;tunw@M%-E$3GLYw25$ALo1N*r z4_esO57Rj*$uEwh6mo81J-ru%UFfR$4Fa$P&KrKUp~PR_m^BRd&Xz$pTNcH-q1ho| zv$n4w18?b?l>DaP1vefV(J&b}3i2w)Cd9Zf; z7Tw6FhvhjwM>a?e=z2URbMAD5c47A2Z?lR_1uMJcbWJ6UM|x39*w3k$fH|RE(Jt|% zJsS=Uk)!!g7t8Yll!InBv9g1Vd7Bhhb@ra@HFCt9I3%Wmyblp+%@yJyXn*~}R1BHn zE{O@i+Q;7kae*VgZ?1^d#!}@zrUWZ<`XAhs%mw$WM4(7qjsjhH3&x!EI@bUqPm?}< zMpsB0z~;tqm;TgzB+T?J0jZ-8Q;vtnQPWf@)(?rS9>K+&fxEH$7VlQo<1ek31*hb) zjrhK8ZE7iSf*e3A2S5Ta*dcAo2MdR?WH=qz(K6aTDI(S#s^4+ocqYNW7)i6) z{^Hxua{)mNcd1MPWu-jPU)TS*7EXU@bTaax2z?OyI>?I0BIaMN+*^!~kGb|Vn2i>i zY=AS_ZK7`T`IX!iV>%IB+aX+=;~je{6x>{qaG9#T<2rJA$p$T|_2boMKQGdD$kXPK zXBQsKVO?qf2_#s{E;i=#O26$h;q>JfY8DOE7okn%L+3bMy0U0!ctK0_Th=v0NnZu2 zp*2HpJ658hfgwtEn{SL0y4Nw@A!m+LY0NHMG--n!ZnZgzwUkMEc>m%`%TXFD1H-D3 z!R1kJ=gpancJp=5YzcLl{nEB$F###HxfBC7EmeJEr~<}gs|BU?r1#uK9?gWXh<_ms z!(BM1-kD$+7tFgJ(M+q4SsU`+6Z5EABnVT11Nxi;`ZBDkqaCRhac;P~>Jq;gno1ki zq4nvLpkxYOaCO5c@4&dLZAC_M!Qa!GGBzYpN`YQMH<<2Uq%QVj3#ny#z9M9AQ$>41>YJz zjyk(Skj$};;t>G?TDaM7SJD3%$`;q<7S-jF5)-e=>4%hiwQEjnKDIH$wOX=X&%an5 z38Lq)zsv+zpgATSWWX{pbdttKb=u_Ox`hqnM;4_O zGTV1%&P*zCmyP239uc8Ixx;_^BQrBef;XPa+fu3FYx0uYH5{*<^nY9K%^_ z5XKAN!+mD4KfS!9b%LZxWIJcWD+O}b-PM3FJhIzivP;JoQAf-)iKZoWF7H9}m_cY$ z;bgUlZBWT<_Mt5H&u65_2Cs0B;G{8Egh!>P@-JH^1M|6|w+YZ_HPKnmf$3y}u3&6s zLMqz77fK=!cb!nXcaYWbme~dtyp_{?=DXnT*F3c2dBEDzI%{H>E&t`~G2E>q?{8gi zwl!)+bkOYYwNPas637E{Q{HI#e4<1g*R%XFJe^X`Jj2G`=rzE!T@-WYvHBTqDNc~; zci}w0^e?Wja{CqL<((_OV~CIu3lAZRzW^c&bV;Sk-;uRi;o8q(Is4t?pw*ll9wDJp zIJ`PeH?wu!@nlc^t)~41;!6E?#h@!#z(+WNN`6CbkrJDv-P!ksZ|4*}dn3{#CZ}IbIOEu? z!LA-6{G8}3l__NHzmE-D}A@NYZ7+l@Gc;|Lks5eX61!5XA8ZLRu3Dax7?(22BiGJqUvxZ4;yJ&vQ!qP{t^_#|02 zYQCy!#`|v@GW_QAh7mFP@!lr7b(GrJDzHQ;?=0u^H?^R2r63)QIl)E zrFUnC47~BQYdHvP`l|?4pebQX9FVi>b_fPm%n`DUvc~}DES_L~DET?SH! zwW!ylzR4LNho`mf6y6qafp76YwNfqOe-4j{IrX0W*VH%tRj(89fBAES@9ao$vN6zQ zcOR;n*~B_VDtM{7J5wA?@Bh{anqgYJ0FLovQxV<~ZGB?mFzZ|O+i_4So!cSe=v5jZxbAKczKBm`V z_}7i_g0V(+MYIjx(xe(94>)n2!M7<>2yRycnkOpAeR4EHA@XsbdXcL5q4- z@4rqJ_7?S}zKkWt*$cvHa$<4L@)Gvr2Eh|gXS{7PG`Y{h?3aNIYzP zSf#JcV;&IgMD{r_68k)q&jEBPEJ@8EB}W8*Ate0j`n4L<&E*A&NA>sEvSRNK&r3$C zr{l|R_Ha#eOe>|a`*6)?{U)UYU(>Wh>Yr7V%5t+Z^%30}UwU{O5 zC%8$9`Kc+jm=<@?SGk%y63fiRQT= z^C6vl`2pz1N4J4vq>XR6q9j%FMIi{S4_0pS%}F(u!sEzD5YU+*LwV9A6~zO?o^>WoBEkIM z|C6lwpWFg$;gO%-va=q_G(J~$ZE}sol_~VQmi#ATxO#f(b#S8th?>`>pP`|(ch4Df{!+u5P9${&IOIb+ar3=u7^fvFcf5cv|<(y7<4Q=3(Lv zc!xm?FB?=BAREBi?99y4R!&F6ioH5NRbxZ+4%|Hk9P~w|qac;sJ;MNOPreRsa!HEDC87p!@H*WO9xQQmdcz_r=9=e+sIf_}KqzLq~$rO@bka#&7T_xsAn z^9s9$QC!|m3+d8S;&X*(Qq9e7P)#>s{x~3n*>Ef9`+RKtv4&0FiQNeI5 znD=+Y4VmGt7|c7uvt6?e@vrF=Ey%E61|v4AQolkJE?x$%-UbJB%mQ%>%~Fcryd z(1xrp;m=3Ba&y#&FHlmF&c7S&3SjmBosr@T7-{`pr=bU$F9J6YH%%MTQ~gGHz??hy z?su=|nM6h0EL^^js;QuYbgE=n_=a?1)(<3d8muPGJUe9b?k?b%JW&^MGS}<$H1~JL zS)3Yup=r4**)Y2D}NRr;y)eKx3ln_!WD)CT0AvO;<8H4Mh_M zmJdZifZ+uGCa$G<9szBTXb#KMC}F@TU@JK-F$F*0t?(|sb3=IB+SGK}8)Q$Xf`5Ebba0)8~ zBvWddgI4H^aQ6ovgTd?tMp%Pi6+fB{hG)Xy9h&JN? z8zJu+G{Ubrw1YkwDjwA{8@pWf{m=G;qHqY0iMSxuP0Vd_KY7 zS?tS{I;{8ptfU1d#s)fdnC@Nv^K;sy)o} zZ)T3+ap8W+l#EWA#30-rk>0S80T|}ur`hIvRX%&wPH1X>Fdp3Vep0yFt2*hd-ro7I zN5Oeq(zf~>l*%=EkNq}2;X**dd6k8jL{{Z6wDI^`%q}SHD|A_8yOQMB@?8}cP ztJP~%|8@0yh#ge{oW$P7bnKh>-EIJcvsYXs;Il2a1Cx9=W@@<&rK$qN`h-|5e@bXW z%RUarEL8QX8_*G&Hyrnsv1l30b{sCuI{Jv^$9z>dR5ztw-B2;WWsga!^75b_i0bc~ zAD~4dF8>JRsuB)|c7z;Wp*4(BjYzhfTzi>)0jdjV0jg5;1@v_V8^&c&=tYC?`gamU zj(K4Xiz#a2^QqI1(_g#>XvlDEtsdPD0y$Gxg|)i~H-M*79~S)}9TdIqX16et*|%qL zPfjvDSm5kG9CplShgI0RL>#;WdG>7T>U5Hry`p?Gd~zFb)d6gPBvzfQ_BOjgBD#PcS%mJDTwgz!4e&!H+yHPWIYy(Zbvp zWqmiK^u+Tj27CztABrdixq&VW?l#B%HYlR>SUWJ5%k5CSgxhP0%@?8C{UG(yIVdp zHGtq@%PL9a)MDms=`N(v!J$=omVdYw(m949`Dh7nULsN=*=F37!?u~styhYPPf}F? zG<)qZ?Df}jRA%F&fqbScN18I?z|u8* zNRYD?z*?e0h|V#gq;%(##K}tGmUzzMtJjOyHJKsU?>D+1@P~m11#91Jmtrb>e11~! z&)1{6t$#0B0UwYO)#tzQGE0$Ckgdg;FG+|>r_Xukz>fGO_vuq4@1u(klL-+G>Cmd4 z=F67ateAyVp7gF$;3*@?Tz-j&3q8(F52vMMZozOdBO%sS@F`9>7)78|2o^ zpixbq4?f1QVNpCT(<#}LCE#Qd7_ZgZiNbPHL4~mb~t~B2XrDs7s8E}k|WC*W}CfB6`p4g zQ$`gGmzoA@mm<}8MAXQ76!&g-Uk~%cBE9aT>y3H zS=jn9@PYMyH`=x_<%P}v7pnVF$1`Dgi1x=&51Aq?Z;{;zdgS;juvX@=9c-LMo#j|` znG6_$O7d0YQt8KN&?1fDQg)$TNR=B2T@l@Szh=#oj2ZqPZY?|Wrz8NxQjy*~udkc@ z^xx}vC{xsbt%zJq1Hl}jM%HZ2gn=s+Axp`AT z-_t9lgz7ln`}~tXfBR%R*4AM$u?IB2pq7EZ)u-x*T1LrfyjJsoZEPfmvVXUFQxX+} z<~A6|PNFxkWZNbE|BvtlXh-kJlE2r5%)!3gMi4W*-BtDth{Oz*8njn@`Y(Eap3(U# zstuFxTxsfB;3*-a-f)(Lv3yNc>c~REuP6BE2wt;I$yspymChX-9@d?Jh zOLosgWFz{Zi%)=2o4sn3kqX`?xgdS-vFRvj@8pp|lXB$FlVDj^XBziq$f!K9zQ+-F zZmc&1V&4+7D7fYn@r}=tPf)b1+;J^ozln=dHu-^&!9})h6q62Va}LYs0c-RA7bkJ6 z-`{5iAN;RAAK#tiygx8Y?PVP7y}i6r>?#P*Dt08Mo@wyke8&+ z0vIog55{yXeag-ES2CPJSR41Uzi-e4{`s`R@vo+t85=`3v>vMXA;q#^WiIj^0n-9H zZ7quRoDbPjH-K8Dzdwaf$DVk%)lqQL+Xw}@(M|lz0M>yW>bG@L#RZYd=#yC(ZEv6! zkYrkpd}=+Sgt3i^%F=lM=ke-}o`1O$kis8e?*13A8u$2rh@n6pKlli_SkzEhPVtgU zhQ%z*55}eJDY63TJ+M3^enCX!(kF1ve~sLXn#-~$yD(R{T-}x_8-2AxM5&%j(hg`m zCWSNuwY)uDY6el!?l)X(J#sR%(xnJm1rVDfqM0pNo1!!!^X)G<({k zQ@NZ$NXB;rR<3*>m5+$SeW*g=Hbi=FD1QO4wVw`Qv*dQwhn{&v-A|gET!R^)`1)ty zt%od}A+B->$t62=39)(uvlpd;yUIKWGYv=>Dbo$7=Ye+JGmW*(E==}%tf4f}2ynMY z=(LgdNYUuxd<$Qi@k<^~0c-elF<_OO$*zKxGc*N6Y~ zpu^|Kq0n5c7mvtNmibLztw-Nr#y7;>xA%oq)+IjlbYoaXw#(XRgk~XW(sziZ)R*Aj zqa)R*>p&U<@h|q2t@#qE*$fG{J?#b>kM0MedaG3Te|J_dtzW~Jv-Dn1o3HzR zKZu$pY%-B+UugC0b9!QKfu(hu<&^v(#_k=+5W4{Xt4A@&4=bM7&ti$$7^}M_r=%J1 z9sTnCQwz)$#rWLSvid@&f#PfGRpzJUR}b#@?+@CYgmwi#Un9Okrv2A4rY$hPs^UhICqd}c{vLl6U zZ+q(wOPB&1)vg%|oCG(p8mPy`v6_TV( z%9N<(rMt7j8!Y;DyjE<$6=;J^5iLOGtO<}{CKt^OPN%b%7uq_r9gKr4zcEsC`5uuo zeB$_NM_3+n)&&5k+Sexi#kK+hT0S-;m2Judq`zbZe@ulb+yX(Sud12#5bkWlZQa>9 zh2qZzlMW#FX|opQCW^*7@{NQm0B488;$B|M43})d_2;ELb}#Ny_AfTtYx>Biw93qc zTu6-{*Qs*fKs112^v({ndXfHmREefLwk73glpx~B8RXcw3&KBi>h zzRFBmde(4p;WeQ0u|JLZzb+w3E{2Ln?>^M24~R z+@Y-w5p!YZ&jXQyc3t^4Z7WZiuND(Rrq8RPnW|$!N2}seUf246u{oOdTp^8{p=Mu= zcr+((IQyCAb#%~LOv7Ub6Z-aBmP6T_?nIf>jYjEunKEDg2?Pr`Twv1*KYx9@8Ld^l zxc5Vf1K`PfJ&C7|PW^*ieO{Ke^DK4Ax-RJ*7HR`|fazsS(XR@NCbPN7%8~s1-l9Oz zdsL=lKSWDAJjz46T(qQ6Jaork{D?OJ?v&*Xt8Exg{Jr{8KJw~(h2N=l6Qxa1I#*QN zBlEU+v=MN+RG2RKiX^46}^r+?mYq<_U#AEF`b#ube5 z==`C-C(>l^Xbmc%0qRhk+gM}Us6?b)&ZZi@8F8|wpz;$ULdWn>SKGcXmaYD?Tr9s- z`+JEfcP&m=E06gxV#A_CBi!|FoGY>si8}vI^LX)3cj~)IbP)Vv)ntRVziOO7)R8X( zGih?|BAEg-?ZhxO<3(8>+u9t>jvcA8t~v+Voklp;dG$0`0CIeux^N2!0ri7fN%o=` zYL-?OU*CDv6r5xM%+c-z>2V$`W;)B{qc&flzHFKX?NE>9?ckLfDUe4YMo1$8zEcz? z%tEnU2t=N`cedydJn&I5C+pF^{V^?ucBA6Ef-y$h?gA|noC4mM)5B<>)+TYNED>N0 z5hGFy3#rO3*QybFA{$JXmdjV+#(V-wphOlR26HjkT~4*`eeWIj_;-6g>EDFsren*L zM33m8@g0CEgmJeH%%PI&0p)&!D9^xvnIdDWrWZl)eMtLT?svf9!Cn^9c)($7caEHRue2rE3NOfMuSFniW)UiC?_^(9Qp z*Mi6TEt5}BoNm#x*&=>0Wq85Hdg?W^r^ozLyT7aQ{;;}ldx@|m`I=>h{0bl@1i$8I zej)EUi(>1=`J627EDiXt7BTAH)VnXd7Cp8Ek~8Zj5bCT)r{QIM$4f$D+M&gE$>`X> zdJ7H02nrKEUpPvPC@R*+p{TcU78BM4lcz<&TtE_>41I9Q;~SW_TCJa3^k(f6t5(p( z4UKr^~-M>QM1Hi!|P z?qxxz5>WHAcfZ&;qeJ``Lb4EslZkNOHb4mL5!q_&`3mS@C{Uh<@s{vF$s~ zXB-QFrNh}HmV5j}0N`Dg+-PBMNuU;h6e;xCUdO57uCfulBKwdR#UTt*i;y&#Zs1N+ zJ=0M%%u+F0|;Z7w~`Yvz_S6 zuFzsYW#Y!V!{7fpcqHX8J?i5^S$9UKWTcqFrjE>SqA6s@EMA}C;^w>B(FX^j6H-tr z>v;Ys%)0N{k?Sdl;Ffn3ZBmyNq?^R0zQp?NVE{q*T7&w)Lb~P3a>?r$K zE$W2kz(>wKkVTEf-9eCZ1H03}6~F0fn(MW#+F5zL@_$t*O5)cO>7&%XL8yhL*BzG} zL$x`y*@X#I+$_u*#E)7^&~};y5e*NF@5L&YJIZO$v^9UosIt`+JHJz6qMIVC;aHNZ zPQ+PT%e&P6mw~Wpx$lZl`HZ=W?)fz%lVo_}doU(;+V=-VNx7HtG3AD`e$cwHf?@$W z4(tT%Bw|G^6D;9%x9T~4-3=-_(z+LawtA6&gK@$0Di6h`v+@6akkUPenKhVpl{|V6 z?`pthCiUaeZO!Pga6BJs35C#5JHPQuNQ>3F)S<;7#Inpp?WrdzgS(ncBN3Fa&hRRy z!<&YJo0#X6IkF~ioe#U$7M=E27WO@vsc@fr7~SQ@xKA#OiA0*rks(1Ru*{!zBr}*m zJnQm(QAGMO%}$YoiXSaK%;h;HaXFF=jo7^FwKS#}+3{%Q+>f+MQ7kbkXI1Jsd#9Xx zSwoU@{?JPTnJ*v&RcNm?|ELi2w>VvjhHdT@`)aAQ(7<_26}7sOa`=tI>g+2^*ztsq zCG6CjTWC`LU1XJ)5cxWja^$h||2<>anS zvGrRrL_@KTh;E}R3p4Hr{^t#R)P(M3J{g=dQ99b@9&OQ3yQAU58Hx@23jTWghl24D zSy>nZ8(WvUt(%12gYnFb#@LPPceu0h^gsQQbI%S>bFho?r(K-!gOD=L7R==qCm&n( zK0(a584Lzzk9{}i+&W8Y#iL>_ES$w!5b6G_BG9?*mdy0l=4@X{{7xU=CzFU-!$9S0z;4$Im=Q9x94+ zcz=ugBD7%MZ;V{^1;%`1v^k(~$yc^ayK*JqWY>z&UW3gJXugSs6eW=0jT75D6x+itPSMCKBsEi zb0ZYlp$K*;o>+SDaQqlhV|NTH83k)ofJ(fe6G=I{_ZO|H+6y&f z;IgYbkTHiNBq;O`@8%s2sDmzu%y~DcWlw0?9|JtWxe{-KVwhi9j9~kj4 zGTrLNW2v6lX9?EE?65MY3JF;l5$JXKMy_96cX8<%g3?IlR;#}`x;gs>m5vkY1EdoF z^QiN}_RZ7AxlzVfEYSDlN2Eu@FmI@~^L^?qKFph#PnG=zW@`bM($?oqZa&x$hr|)) z%Ld)Zq)AF2Mi`Q@K|h3hgGz#P?S7d7het9y=FDEOL!YCFrT4J(t@?mUUd1y#mLhJP zHyzq#U6Ekt?&heyaEegr&FPMHPziN>^=>q%MDJ+vR9ex|WXYHm_0Syr9gx#Q%2wbB z)0$%#RZfK~nl6MqZkB!Z@4eZpkBFONQrGK!e@^I!Y`qgUXaF)9^6k;MC&KVi&XK0L zPNUd%vaD`;C5m=)}u-3A0nCwRTZ)riOPQINne< z`9Lkk6>cS#A1XNtHF6uQ&Ly}TVapC3K@~wR33}Il&#aWQz&mQhE{wd&l1;PvVGkE} z?wYDScC{@o>Y3&eQeJ1eceq&w26Z)lMlXne$U@v)=8CztKPfQ7%my!%3OJIupZq`1 hE&k5~jZKgEYqy9hU3hpMaCQ?(RZ&BsT+Te=e*rH~JLUiY literal 0 HcmV?d00001 diff --git a/_images/memoryless_5_0.png b/_images/memoryless_5_0.png new file mode 100644 index 0000000000000000000000000000000000000000..d0fc0d572f0e4b44550b7a8312f1177aa1bb457a GIT binary patch literal 12954 zcmZX51z6Nk)9;efC7l8a(%p^Z3L;1@-QC@w2qFk5i-4rS(o1)TbT>$MclZ6{{hsIj z?t6Kt-E;nP&diya-^`rZ@b~ZJu`$RoKp+sdqJoSX2!!AU{C|ap3jDsapppUpAiGK{ zs-pp4erRT2fY<0w3OcSJ5GKLX55nktz#Z_Wpqs3=+XqJrH&0_1bCA8Uo0FZRo1L}E z3lDP_S8GQH9(HbaZq^r8Zf;J(931}{XLoe5LZHhId=YJj^9_o~n_~pbJ@ubJ! z&Q{s?4MUzms|QOY9mleM|}a+}GFFa`ANJ zXaWKPA3uF6)n?TQhy)hbh)GV4tgaTY|63u&WbP)^)?*Ft>F<9jB$OBv6T>WVQ1*r$ zoL39ZW#znw+FJa>jF!f88T-`4|v=S0&49dS05+WeKp&5#?3|eL}Kfk z!$nogH*Q{cIk)+PUVU)|vv0MCpZO`%emrBDhEGMMJXNHh$Zt>1pY$yufwazTTRI>> z+}qpR?O-mk(f6{Tl_2t%M*?mn1QwF}@QpJ2nVQ)!N z10TFWKUsC!N{riwA+9HDio$MNEqNbux(5c#d!bA+GBU%!w5Ep}TaYFkBAX`ezqQaP zQoZ2AQd=8EJ8M5%#bq`8!sl$Owfj50koSq%_DOfQ0>DRmp-ws2HZS*~(sqj5>&S>f zDM8Z8>UAuG^2+7Rl8K4Q)@(J$_wV0jyV?Pa8tZwI|vHR=O_#L%Q+$rj12*1(xg77 z4H6ZpC@2_>WXL|rDWao|K3cSGGc9m%-g9vvfy>nEpmtL*@Ilmb-koH5dAU%({wG|w znt+BzWp@%z#%Dvg=y!_ke|upLFkv@a^-b;@IhV;Fw4dzyy*19O-FisxZEKYrjRMW5S5*%4UnhTYnRb>0 z-1~G!(+o`+1`YsX)h;zb;b^*|8~DX%3T*P~=BCVYh#EAQEc8ZB4wa+%jvF}q=c%cw zce{pg#zd2Pysh{lmkV8BHyw-RQ#ydXmHO@iFWcpvx*G zBm|ms8&j^St6SOG0UH?^ZL9jre*OB@Yp=9xy4ID^qCD~b_EM|vK3}6iyV8bezRn%# zg@8aj(lZ=RrzJ^?!DK}*FG1k&ELQ%6!z$VJYD&(IzmfB;+#WVFii@WLQb2gGt*rnU ziGq@n+2KMX7L7P1C}}|MbYq}raFBt8B|_>GDgq~9CINTHU44@@08dU%oFM&rk9EL) zfRh|;@ID<~Op1}qOvqPwg6RA=Vxl@ zFk$kKoUzNUsUZc`&kt@4BsqI}Hm`9N6&1l2)5F0)|t`3Yc#G=E`Ulr@x4je7F>j2&irSEcY=FR3{$_&5^ zN?&pd}tff58lzJzw1DNm)Zp7SoH_bo@`pDe{#cGBmIb?z)-xHO#()#X+rbT^lWD#Zqk z+SLwJWMpL1cKpPw+V6c{LcZhE%N-rmAO2i;^nVSbTUc0t$sdvQ4Gv~|pKc^5F^J!v zeX!%#h24@1xx94U9Au+96$jX|wzeiDA>lFaC2YD{!4!7i`IF?>q~id$98CTV7(U=r zzGr2<1P!K$bUobPZcmreZ*OnIf(-zG)zhO42ncvGN8@%>koe=>X+e<$FE11dB?SB( zsJo}Zweb#>$xfI?4SUxAyUhN=V-)lrSZi+QDI+Ku&XlP%JM-%3h2 z4FaysitX>m*%dM*cTv#+{L5Az@+9 z2?*K%gt1;|5GroEVSW>eMGHU);Qo0$_NKvUw<(?rwEX;lKdZwdChi1u*Kqqn`4{hR z!{z|sX+zEjtfOxF=%XISwBdSB$BV;7!yAVTI(oUtr9qLyER{5I02DZl+mKycT&}t# z9$JGCkVrX=K#on<3^e{{_~_{9PaZK@$Tjrn$cc`gUZ>mw8vy~KKS>}u{5j(j>iAq9 z3^e)QmO$XF1`P?@V>!>s$P@wSd@HebceNVreKsNwyg7t&s8m!DP%zP(AsgWZ*h_e1 zq}kQc^4jL+X$ulAuz9)5gL!lSHlK9T-}3`eep$5btFg#D8N;#Zpkd3U_nZjL1PP+pIfVOYM8%aG7iK?W6LIHXWk~kC! zxc&wNZaovbk~zwOdQ5-z*ND~11}XG`c#kwc;F(YTZmCWqnd3q?GnogzG_TXS$S~VM zLVK5MrckJ%2MrfMIfPUF9>*=HJKW1DwX>%{L5$^848s|AbFuHMvCDkghRJbsvs-Lv zugI6rKIH-52~4lvQ~s{P(wW46)P6v|kIoSCfw_2f`A|zz6GoAdD+0_&QqrdD92=tS z;t*kRJfoNK*EG1*bE;@e5%(7s^Eoq`E-|*7udA$p?U$MNff589sR0l(AJ0{F+nfGQ zV(3@u)@)#45VUHNMf)c7N%aWu@ts?Rg90CK7XfX+BO~jHB;x@ki23pZp7tf6Zn_S& zp8=!NxFKh@=@9YQ)7}_L!)hpcHu=W`SVmsNf@=4pXvTO)NPCGTzF=k2%> zg%a0m&*7DmK?Y5JwUdxt=he>b<@T`Mwl?WT0Gt75l2p^N=TFxA+yVdZW{|qNsAXZ14NQrQj2yL3 z^S{u!2kgfRUK#Lky?Jh1-00D>3t(Bs!}*k9(XU>!b=Ch{Yx;dt%rtV%|HG_S9V>2@|bpYhRDmBy}#V+SE8I=;! z?fH|UA|l52|5g^9Z93hbPwfI&2(bCEU%hx1L;HsL-d}S9CD8L#khZ>5k*2(a+bW8z z@hO^F2-{DZi{!Q6Qu~ob3SVqAbZ+$klg(9)@}8roqjPe1?=UIF1Jp%|;{*`zSeD{* zkVDh81;5?2r26=0tr&TE`S{tk^PIE*6CmL9^6~-=*EpMkE)M4VT!+NuH8qKAYir~8 zskK#E763>(xomz+0kp8htcQSHz@Z-q5Q=io?PTE|t}ZU55&?eJ`=*_dO#TnOJ2V0g zlmJSVl$4BCwXEM$0%$l>Vf`EkUTwEcF)=Wh?}f$*H__Z;(Oq+rNbCG*pLh#eVTxOM}5+*iasel~}#YdasM_e$)A+E>TrGyzrrH-^pEZsNlq=YRU5fB{J?B6@5E9G zUmDGFT`zbcK=7}fIrJ^-;^Dg|*Sizxd)TNIHAn&4#BkGcRG)2)tU6e#PhpCC_l4z8 z+>d5P)uf`0!+}~vs`U2Pzy7{6=>eN)xm1lzpfN6QY{&uj9#WS}1hM7j;c<3#thj()cPB-cXhDllVVzlsFJij~%tSDFY02nEY^gO_KL+DhsK7b);n>sg84)@r@SP>mO~9CasEtiKc7+Z-%m;2PMn3Bk+I z6tqr{Zp*TdJfCi#n*H&uNDhtG=CdoR&wJb>+YT`UU7gs3&g};2Ocny^FXWra!4BL~ z@gi19&)=BpGA!=@a2@@OH#j7OMD^kX2U^8bIl|wxz~0U-jkdYj5WJ8@g)GlA*zkS6 zXNFQCP7?&Yz0nYiGwe;N>~cBw7usqf1SXLdtYt>2Q?r4pF{?7gWkG+b4*D3v#y!D# z`rtf4*f?~DsdJz7pe%O09K}wlm$QzJHYArzLr6Lr^+&~ISjFV$7HOkB z*;%U=T_N%qUxb`slbw$^nP}b9U><=aKSAaYCmRj)vlmNViMKtk`g(mxwve+xzCnzY zlNS-5z+UsAUp2rTlMt0i9QyZFh^bbw{a+h%y2Z8BMTS^H@G1+LfvQ1;1PQKOG_nUG zTToF;J(wx&Ep^s;7-aV~++AH#vW1AmR|9g1gd6B!B8Jl8@pCUG6DaAp=7ayIcF^hk z(qNKsiDEwnnG%&mSclkFgMJ!B&zOW??H8sxa#}A#@Fot*f{QBIQK^wM5z$-X zA-jOU-+=SN!2G`GTLsX+NOxFDQ;?~Y4`9SMv^9+1$rhKb`AUs;x&A4HjTS6m3$nZ3 zauV^g(QxCI&`9R|r{)0NVar)NO(Hp@#kc;$)$SIj-#nX~)+=DO&e~!^wt_Lz_Y4d^ z4)o34gQHZ8fL;T|ERA+_w_-`Fv9uOb34)QfFcjVBp0G@5w46~Vz-mstxi@!vskhJ>t3)Y4C!v_5{ z3|@rShR)j{$zhGs@<61tyWG)wCd#j%t4|y9+{!B$2L=bp5f-;D_QiQ8MUm=%$CK78 z4<*||ph`FA(DU$b&D#2TSH|(Joy_|m>r+|AsKSC#qdXU`Dh0f(79Dp3^8Yw5Qf$XQ z8(eK&QC$7=d5&--x!5GO=lciJ&v@tB?bvQw9B4GSzVS|hZZB$4I{X@y8JdBDo*}jX zz{)(fthbEWy>zu^fM3~lZgSh}klbo=kM!zD)JL3>)ei^zv!THHg(C0Yx~5>z%JA7v z2NywC+ecP)Yq*YhgBObWXTAPx<%i&{E#|%E*5{5k~_!H!vdA3meqHx`VBmua2InOJ*0| znFWBqV4(wSos|-D8I~|Jw^&W&f$bU~X6L3Mjpi)-ayW3xmfVag)4bD1&eM7jP~&cK zMiO##+8tc@icj zZj$xxVNW67GnjQeUdCXoB!@~fehBrdIB7%mJXa7sKu!nh9LbAA|0AWtnk$Pcihtp>@ zk3+hcVAN`0G4(QK#1h15Y&#V3QOF^@whWu|PIlO%zqz41O+M5`kUmwc6NodL;e#10q6n#RQ`+14!RDjPMt!NVq&B!S|ph)&!u_szd46NHr~9Q~vd-y`P`HymPW0WanL3Hc_@>HOpGq4scXAG3g=?m^{iqkCFz zSIo#FB{bTX!^PTbGFChwP4PY+|3)D#?f!nlACX1ie?tj9$fk=Z_BjKc+jG=J{hPW` zF07}?Pppy)qMsFv^9wm@OGbTkpvmMFhb4!XezeF@3 z-RnA7vmRh5i*! zigO$(IJ}tM!R96 zVhv(N>=p($;tT&MMk4#TZ}VZ7MG-jJP^n-wM)8Gj-<7FY@DoUnQ=^~l5D1ftA{lLN z&dmyhcp#+Sxm#0iVnlocL^>j=kgi2ijzZ_~=?{@MX8Bdu71}$ql+FD#98yo}%& z$>Mk4J%bw2R+@fm%`(hUPyC4Hr(_UcFZ?H15I}vL6%kRSJ!(kOHno)h;`2XeAIE(P?7Y38a3lE$CYRv~>;Yj_i@+-eS)y#tD(t<+3gvoseD*-}H@n>Yu zNM|cikpAAD9ukEo_YMlw2?IbBD_QKo@I3N1diBj{+GCX3*Z8i_AI!>6+sXw0k>El< z`?WNWm60Re<{6W!u*;AtDpV-Ovhwb@{UBCOi!sDWjbMupc1dsK9A>zPY_t2 zX{|hXKgAHL=!!7UHdN?^OPIyuUWvRT)lUHm8w%-DF9i&1wy_oFk#rZHEj>Kghuh45 z*^KY#(czH(V)H84U`+U1Xw@5|)!FxM3tYd?;NJ;ndB>U;*xo#i(+3++po(LAZ9z2bw(5{-vMp=#oZ=FlcuGk^dd#CU76P+RHBPm%K5zw_!&;8<~Y) zMatx z-Sx$-V4+uN6@o#7D0?zIOy%?@U*<_w#@{@}x#@d7qd=SsW`cbTI&Qg*+gD)X(_kVh zfPPh0QE489q^J}u3_s$Qobo8>wGPaGN+g~CdO1mL)?q-VLL#k~n2%h?Y&p|mis*YG#f@ZOdRRvCqXhL&sdZP0$$4GrrP%>(G#(%V>x^gG-A)tFYj0=SP4&}SJj zwKz!?RLIX-8muh63T48Q_e}1{37|W9CcYvFvzrIICMnPJ!R(f@E~kml4FY*0MeGdO z{^V~Rq)PuCUwA&U57w{&6Hx}V4AB$R*Xxj8i8BJJ2EGx9k)o~O@tVGsgUo^$q0MN= zi*3ov6(wmV8!7XG$=|!i4EKKo0J_FpYnN^dj|lE^N^blC^2@7?`T!Qog8JlUW7=^7 zZp$+OU8ifQUFxjz(5Msp_g&2;Ax)u#nVUOs=#pgaXCy>KFra3F?R_A;e`b8;!kyKb z`M7CMJv)`S?)#40*M+dX?HBozRT%v*ukWtBikEOgAA%*r<{K#>Li77=42KHqoRl6I z$P$5=2yhm4-4nB1cLd`|Q#5HNgI6!^=Gf*WIDU?ob2EkFW%vf)@{j9DP7OR9gN*D0G;}^XQ?5_z%qNbUY(v8YUKmw?YR{ z(7zhsbfz_9;9A<$O_;f)WLZ`7z79hcN>N2~MnFHiNxygUTC#LHUGB8UEz4aO@GEtu zdDQV8-s%m(bN_bo+Bm&ys7S>=6xgS}cZO14H*F zGDPWX!(Hybye_M)*QIt69{*qn72vB_iDB#USw9)NpU@Jaazo)9k0ncelqDJ9rQ2Qr ztLj{_4L7g^NZf}PcY3dGS{t7vv+XJVBu0?tl>vqmXF8mt>XJBu7v~4sdZ><_tDUBj z*+wsAzo050Ha4AF<9Pbv)LJ9o-;fZC4r6j?ksRDuVIDJCek*#%x%8$A1j65Y>IFbk zjd(enA{F7lxyFIl4E^bAQMUB(dXClN;L54o?>(~&5?$~C>y6m9;Hmk3K?OuOwK&`D z_xGH*dhx$)omp7_><|&%&C-~=J*3v@RNl#iqLqu|bfQAM!fve<8f~gg#7IJezPtx4 z6RtMAuDNxIY?maz)TE+wW~3rqd9=JdR9$W+U-o1THZjO`Jme>=6$j7y9zz!Ic6B&A zleYN3dE1PzTNF#Y)F=P^$qtQXgVoYK`^agH}mwOc)wuwES$c2jNku#u^Bt8O7q&^DuxE+TT6?Q~f^yve2B zZ)cHwLtp&uY%|{9dhU4HF>Ae>5+*H(iGCK^Vi)|qSTNLRbfwmP6UooT6%!|GgZp6q z+nyM$RvgAM*2`gvAA_sQ3x~84NeD@Yt4TffBP}790>}G<&!0f6Mvh<$TRFnPY09Kq z5jM8dY1%!S7ZpF$4m* z5JdgE_!*Q88aF-tzxLevxI-4Y-F6=ba%xXA7roZQyiOwQ{RRJstOq5?iXUbsk__w8?@42AiYtvn6QaE#PQLm2XK~E1jk5&-T z-^Y8ma+1ado>?)e3fv-T6swJ6!*APiwP>yR=r8ZW(+(X5_r-KiT5sM&`uK|35{yL_ zz5jv=kV(a=V0D!sRjBo%hpCe+(Hj*`LI#~(HB~R$hnriM-Mxk=`U~)_{f><|`WZR^ zqx5nLVC_;#aLcCrCd9g;7cG5`DBPBzc&4oT=W9KL z(_j5HMJzYz#fC7DA_gU|yq016pix_)%j8afL~Mk*VFNc*1&bl4R|Xg|o#Q)UI#sOE zb^r-O?*-ZaMVsP0FXNI1rK=T9!O{U|iQH#7^%A{?&)-}bUn+jQoAdTDlkb$NYB6JW zTziM+41=o8GWSj~qxdMtNP5!c;J#UL`EpOnjVswjp@`M+-SQHZz_%>n=mpJ#%NxcU zgH1;_nDn?W!B&RH^jG6kCGAI4!i{t>q#1JlPZE0$?csZVsGe9)Xc)qO%2lUEyTgX= zrdn?&>$E4JeN7@*N#VP?t3Bn$t-|0&SDl`4Jpb`2=e1frw0UuZCGIHIE=#+NU?^tY(6`YyM(aYq{2y?0tr0) zQ0__gC|H*t)v|1<0k;y^?s(P z*7Ajx#f!=nzfudgcw!~0;h$fO3^NOW$v-4X+`DFCo!scNI$IAomhHGZg7FL8DSBt+ zb<1&7b#AY3cF5(Y3D)d43kV>UY6>liBtZO;sUq53-Z< z14IL@u)C_;S`CM-k4p}pL94!J-(U4irYK8&sQGQE5*o(Wu|vmb9Udx=s6TE?X?ktBI>g;SqRiKsMB?3oih6fSH33HJ*&9Dge@lxLGB%Cq5$v)x_Ng>a;#y%eE6s?S;qg!69@ zzW`Z}2ZF(GB zK7;^&FCuKBW2xRI{k!2v7r(1GUwB48z1P|+(~-|vwuaO!ejiDNo%Pd6 z@TCIW&@W!mO~;pPh}Y<3eaOlc2YLj@Lsp~(tH@?A10_?0l*&{DF19t3dzmbJr8t3o z_VDs$`#(L~6vBT=TahS0mE^s8xNhTio{#Z@CNS!@e0SCSMsMTWk6?QuCj+n)dbYtTo_ZQ)D?DnQm%#qdi|pmO}}FRj*3Rff0!<K^Ey>SO(NWAkDwIw2LpDFU;?=A0$>}**=%NYgJ0=^{2K4Vgl2oZM;sS@f62bem{ z;Cqjp48PhQt;9t~E}f+uIeVm&1lOCi3#b8}+H9gAX6+VT z5=`10r|uTV@km--Dg3wxVLF96{0Z>o+`dvf8FBrL5y#@Il%l{R&<;7F3G3(x>9A{? z=m-I}?NS=Ev4CTK++xCS%%<~r8^~);vwEeXFp^h`tN-!7DF4A$SXv957_8IcM46V> z^&0lm8ot_R=Ycfbv^I(9RlHV!m+_l3v1P3J#J&NMHXSKs!EL+l&aR8}QDemEA<{H= zD`QMF!tNJPnz-)Zx3i2|PQ9ZLE5k7!l=wIq^>9^ZMnb-?9co%zDHtDKmDUovsu?V; zr=Y4_nhCzRUi61D3Yj4)UEgXnO@7rd?|wzDFbbhTd=F+r?nE?4@@M5V6v=;7W?IjI z+7wla>+BO@(k90y9af4hjM3c98kS9~xty}ddx4oSxP;tlvHRM zU+5B2M?bw!z6UX#`jw9JTcdaxL;cn8r`KXV=uVU{L%%cq1%9a(6rh6h3quLTY9fCk zZju?e&FeG!xS`M9tnYm&_}WzrA^y?ox3Ky;WyT3=B6|RhYMlN5Y7ItKhlvAi$QtVZ ztEADv5Ax ztp&Pu;foNwchZRZXtc>s_yjbHx7V>T!$yNMwXU54(|^3JPX5ub5h`Wi+U~?wTw6I= zH;3tsg?VQC8~PQVEA*}W$J88Wx2*%I%?@8*OvHaW&CmIUC3x#VNKa#Hd;(Z}C%<;r zNr4r`P`C@oU&zWKkZ&YuEfx{GQxVw`cGCN2C=Cex{@vHCSw!Uisa&=@RF(PE40MWDc0_XTpnUm|Tw zg(0)@r`V8*2w4$gKAmsVFKRR{+h+liu!YYeNyN`JTbwfwGx0R%0kyO% z|9}rUuuAc~$_0ZHf1IemH7Q_`a;REsi*k#++?1LwR871^(g-JLvhKZYin*Nz-yn*U z7PuBfEx4T}2-!Vv*^VHNMiFPaet2%A2kh7%^X%&eyFDIM%|6JXh1irO&)E?n5Oq(E zL@nYk;&w~dt2fFtOfXT7%k&$ttv{j{m0+mp!Ll9voY}M>Uqd-C4LNts*2sOYq6x(( zZiC%j4$Q={WV;ktISCXda(sB7$U~ziQtiUPirr2 zvX=wkGKpd1CPpGrJhdK6S!^Jv8fH*yP_r5{nHTjH;}J}4CxG&?b{@hu^eX-V6S!*6 vb$kP~EYLE)2xk9pqXAI&zdl`YCvo`K{~`18duivWIOIw^c`1%a>$5&uXd^Zs{0qaalFJyg@t0_tJnVh*x5fjZeaLhYDXAHW-P^2!cb)H>%YH!&W+lS#we85bN{ov@w1St#h|6s)!JHyYpc8#2_%L) zi0k(@2u=>!2_yN*4S#~k28G065^8V3$%zP)yIQ1oS)M^%~s@gA*Ly$*C z3Y1*6UguSHQL&w=W(g;xrEQOG0A1-KyDwlg)%0Zldk8`PFb8EiR#sMeHa6^v$z3S% zs7y$Bcw$nLjFpvDSq>!BL7BtM-289p2VP;fosbAY>aHh3VDra^dqiua>!z+m1)QFY z!0V07MP&EROB&#-vJ8J@_gjK(59G#Am>7l?DOXpXY_6kpG9g!2S40n=e(*!KMuTE( zXe7wCwEdfKJx@Xi$%*Jn`&Fcc(+=w?u?@gvAavLVbE>1Nsv1hOlN{?A zEEPrFdhd)fUcplXwaDZfYf#*&+$2hTr`005-55TzGNesUPqWlkN;e7cS{*94KHhvx z^ls%Y^rsevMg-jN8j)Oj?9coO!6btD4c6JuDkv%zA6l^#a9fHP8W4M%w~=Xik3R+b z$$>|;UUSgrjqvK&jzPLnss3U_X}9`l3<}Tuxv&CkA$*9N!w~-D%e7{v} zFBkTle;XPqf8y6~{F7Qgu%f1>l^>#MDc&ymQf9u$to=4y^#TQe(EP77W3m#=*Yr2WPv~|B zBu)&iy~cWUs@aRz@AhK7Zq`A>0?%W%&i?5${uuxJo4*xC5~LiuTC2AYyGD<1y}gC~ zZqAmoeGZV8_Ujid*18kEzije2+&CLmvmDA6qY`on=}BZ6ZSruuIos~caBRhFpKElb z5cPa@e{;UN63w;Io$wsQZtyc{xifsUEg%3`M(4qN6I*f-Ah_p|akl45WE4AiJx3xy z{JC6=aYqQ|+1B5Wiiym}tK&<_>>oPf>7^A^RLt80k=EANmrt)7_UmR}E=-Q9N%-+B z5=y9(g>$r`&oMIa1_eo$!w$+nPL}BoJ?34n485;6;}k>B(W%Jb+3>t6Uoc*{BUSHjvD1%Iteqh&ch#ww)=g_c>X6Up3+W)%Gh{t{J{lU*&PAqKf#=Da~ zMMXuEhr4Tb!{*dl+sTnih&&GukLPY#JvomV1`v4W+am>hjtiN8^W{$mMGjy2-+7*# zo>K5x;RoD|<}}=2%z9l+TQ=^NH(l*J2J~^%&vm3(X7}Jp+_k52L*CarP=gi&9>sD3 z9)!HSyr72*Sw@dHlcHBkA4?5e(%1V^P|TUG^0mu!xg8foJ32e38=RjLv8a~1s}=#l zSLM7aN6KXw14Bqn%KpARl+3Ujutg7%n&>ZiTEE{dmkUZ6{PwBBZZ9hwD$ z<9$v{?7Y8R9Ae4wmD1884gi+zxt1W`uo^4UaJ`Y?bvf^$sGwjBCw0ph@XYd8;~NF)?Aryi!1SGE)EBcr22zE`gZ2na+U z?@oWi;Z$~jrtM^Tndf|GI9_5(3VQqXW{%qX_v9ufCN(zWFw=_GmrP7dbI?&0cB59Rl)gV< zIL}}P0S9x9{aGR@yn})$fzRb*e*mT!6y9a03Al6C(9j?UJOE**ZeiDr{-{j>z(w%H zuL1!d;(>0x14TtgBSK|&vceJw{20pDSfHFCc~et{#>Pe}0I2{yqd5H`(K%WUr=3U* z078I~i3w@A-q8raEIB(n_Q=Rcl1B$_Zf=r8+x6ZrB+}}@opy5#l@9=VvP>K`APZz` zL=h3sv&WhzSVLznDRwTO)AN4}vKAcQSDYjRulKT5tOcRdT=hWEajBMoU`jtO$g8pNe=COChYG1Frdh7 zccS!tC34%Mny05{?QY3Sk2z=bNuz*!HJaH*R~CAD`n~xkGT^d;0tW0R@v;mCtT>8- z*Mh3rcN_*j+Zm6pXnVLxT@S~lhMmO!+Bwm=sQu196qkn3CIB3o`?nJ6NV6e-<_Gg} zgzOBjzl^HK6nkuDy5d$_4p}U=`bmC>bX+v#kKS>cH0biXobURkm>5ezWYp}*tyk~x z=kSsjaLYl_Q)M(*uD*Ko8_+l&4Dnwlg42v2WzD@Lm=)o8rd z65dwRhJWP?Dq`ABl+tr@#!gMWZ@qz) z>H+Xmn>e_bFOTOAq^HkJN}%q(KE&AE5cttrq8i{*$nAjRbq@_yT-4e;H^y$6{SnPN zj_;D(ybUFW>=?MX@LVniNC4qgsFov+Q1E<{$LUEQ*Q$F{<6c!?D!)CYQZoB;^^iD< z@zK&RF-wtKwcV{ zK~3yjt9G}mtBV)#VnC)?iU72R<*;X-+XX2Imj_n){mTC9OF)W^sybS=|td-=RxBp0s%ed-N?}KXaEln9UIdUf4p-gBqU4~_Y(#J=o_$> zr<8mVhi#9D$laT+P6MzEA>saP{YqRWSL+pnz=G#<0M*1d1QLk`4v#%kU)z!jEc)^S zy%3RAG1*t)f6fMg-u7e#Ex>j31MbgXKe%f?)0nR?K4<0NP*FQaCM$Um&72yc5}s=* zjMbpfYy_P8FJRCj$k@!?^{Lm@a`^QUiipwkc<_ZE3xg8)by?LpiaQ?llEUUw;j8NI znIHYzFW}^>4S}3zs-2vFDmo5id2J1+Q<=?vM-bF`EIdJl=CLrlS=d`~>F!?*|5s;d z;C>Ma{k^8k^hg5I`MMszIl(QR%gL-Uz`ng zGOuWwbfr{`H&=0QkbfEQyjrD*uT8`G%SZD@eak?^!=#`|fWv<3&y&jD!lBSAYsVth zthWGAQV6*)Mw4-sr;c0z83n}{@P?RY0nTQP zOWQ>`79-yj+auM)^rp3%K!_~5wa_90AgmccEJ*rFdIQG6I>#A{cVYC>VZ#MVDUc1< zioM+*UZ)m7YTjCGljsb?S-m}Mt5`Ow&i)R2UjJ}^Yin?G_61$*F!FE9GoIZuw@>O| zgC_U=fW!VY*iDC*n@c3SclGsW-~W$?YU_ z=)J7FU4;Od;P0=G=(hg;tuWZZVw({d?CRq3a-s!b4(F9!G@9UXpMyquApf^LtoZ>+ zOug;yL8ZMuSH|VLs`6 zINu}yPF?pMrjPpW7h#87{T(?}X>_Hko2SV8xyQ#t+awUda!N-`Xws-`eZeSp9;5OH{3P zreTFM1ATp~9oSr@-QwU?S^jS;AX(A2$Jr11Kk+PG1|ZV?(lK}T%t`tOFL9<&;*+=)aE|l z16Y;*wpLS|_>I+<(PizF$E|IrTHhD-Rm%QtISmaBl$LpR7;m`49`ZX4ICHZC$eJd& z+<{2{r&3$ng=Q}*K_>(sS^%h86t?&neCuz%7m%Y~!c+061VKqE=ozW0z&59cd$eEs zQuvF?pC->^Q}MeaX}~Yxop=)c6J>g8p8pgjwp&{5dAncF%FfQn#T5tSlQz>^mc+-a z3Tb8Kr+}MnEIvL6ywkp6(-w67ZSao7yM8%E14h4RXvIR2wxltB*2uEb&pWgh#a5o{ zxY()>{VOX$;ZUq9Cx-^Bd}z`c23`~Tdk6NTMlkp6 zfCn!PP0j7`60*bA-G13U6z`k4UiRD5AuFA#ENzc6be-Xt7Zl;BGiQxm89-`DOH1q8 zqt1X=D5~V>rJ;zi{C(>>)iTBDYZ7Cv8(co@ljFxK!iTgtsU}SQCsl40R#INoa>`7% zh1CMqCpP*W%d&mM1Fcs`W}QM(kX3*`R?xkaRS@Ib3l0PH(?pRsi$D-^kq>0#$h)Bu+lQni1X?pRARGUAj?oPluxgzb@|? zoWTHk#9jF2iG5z&)_Bm6Ii2(DhGa4C=}U5mvNYC=)MbOT;FKFx!h`S0r0*J(3IdR+ zH+oy3V#`m@t7S7x4QB78y5(|EIDO=GyvVcMab3=|AihKS>UnzuA@Bng+YW}y+K z1PMKEwC8J#aM=@seuWuY-I&KC!=hC{wwB_`VcFlPqDYz8j3`91jMok04o@avHZNZx zUu8si?g;KGM!#($CiC~ds@lNZ12W^gaSJ>@&<{m${dXJgS489B^5;tI8k-tZ8&V$*?m_vl{J-8%IQxB8#II!faIiLd8F{ z$US@}bgQ)d75&e$!a60_!(cbuSDOvyYSib~oW{_3xe}jB4x$ePyh+)&57H(k^uYEF za58wRFwvQ16b+YeyP2*6P2Wb#0l+t7iZxZrht+6#lKlwPe%pxjbdwY5r|aiM_o-+X zMJsUJ#Hu`-6sGP1&aadcfv_mgFGn}-vK$i1{0%gzCbFMR)fH9J!zAWC+LZF@r?v&* zT)OY>IEwNCm&mfJ3SF?D;_Cil5yI|6QOhg;iTh1%59!a4ulX0|+AARIRSLt?nC{W7RsBu?QcRi>7)~Px14O@(dHI!PN^=#pi)5jG%D;A(m zMc)+3ge8A3*$S$-AFp@0A7ew5jcrdt7hHxJaT|4q-S=IIzuaUZy0Vn@kO?2W!Yxp? z!07vMv`(g41Vl!nMJO(m%9SWIgnm)fB>WX61UZ>Rm@a{1YGIjS@Wg)*cG!kUeDF+S zwlf#%p+{=Y<7&$T$UmVHO+yVizn)v$S&0V+p)Z(UF8HZ-0fQ1*g!(6uLL6AMPp%*4 z+W_Dphp48K+^Fkbl6Szic(>b5!7g}aA_%RjMv&Y%f8{w~mcyDl8btXwE|F=de2=V@3usLM@G)>ztDcocJ?*|=P}^7O9=SzaTGy3HEp%Gz9>+5;@Zx&ShYRK`ZBr8 zJ_mI0L#iSqVo9Yffj@nqw{i5JvT6ZwLtCG7nKv;KoX#hHUp)LbY66Kji9RP;laGOt z&LZgqr|;rF8$M?{vpExc6n2UDSV7|qBr5n=H&zSfGC_i!=su-43CHHG;|6ewNDhIS zhZ=qy!**nyReHP|GS=Lsgivy`ol(ljy@p8F?{2BrM4d1CZvC6=Gl@6HS<6xV8ovBr zGCUE4W?)(+Qe+8F2|jQ!WXPE5lI+IdC#*A6$;kxo=sF}M{Ixr9LG^1^`p^Uo8g&NH zP~$b}&xnRVmO$4aGvlujjI{NtR7YIKrLYTR&hqfm*d4KlN9Pa)+yGObZFnlYnBQj$ zB~S(SQx+T@l-Ay`I??#lQwbaZdN-ciBqd{}jLq-f9C8+nT58xhhFJM2(f`1fZAn*x zG%f}u*|n4}E%6chEDUK3pVP9aJx@P*(0YgztT5j~nm}RPg~!Dzc}h%uu@e46IA`dC z=y1>@3q#T4nj&oIyG=T*kcp8o)9HPYY@JnKq5CR@ix~~uM7b-%x2W>@L$gt9BzF7> zf0TT76W18`1rH#Rf{NXUDxJ4^+$kN7tM?`F^4;}+au*T+({LgEj8w2BI0mI4A5zy3 zHnqPZaE?}Be1(tEYcLftQB?WdnNLFGgtqm4WzRoj=E0?O#x<6hmms}wZ*af{&lxmj z%-&uL?*_&NerY#d5&|0-1XN~?9|SLsDq~IGElV6-+jGs6j|xU;cA{BZTc-mj(A{;r zQWuG2OEc9mTs#{t%S(m`Z|5UmsL}$DzVzuIBo#dfB-9vR|#P*`ubJen3Klm#ar4f&8a8PpJz zQeE@`YjfWx-EqPmq2t2HDya%=dV0RLEdt#h`nncZi30Jq(a0EXm3R_wYRp}}_$+F; zOWq~*LQY#ZptDqFz}soBo@Xob61Qv2wp#sVPm}w@=S;0QztE zN~sD6X1?RPvDGs)d(YHE59)vIm_WO29zjU9k3<7VBt}AY8GRSTspiP`MZWGU`+bkkF?N= zhrj=XY{zP$sihE2P~iyUwOGVvH>7MJJADOzg2r}s!zeZKjpZmZBmQwNxV~X*TLzv= zXAx>R)Ui5S?Xg7t?ONw#x?dgv4Vb3dTR6G9sTos*)8X_Pv?ljAZV1~xVP!iD!zYyL z>&vFs@=a>xPW#GGfl%5TKsiCb8PCmjb0tRm>SNcr<~M-&R>rTAa+GAHm^^Yjt$aFn z&Z9@#vW{qvuat5$ZYlQ<#u1DF+#6_?#~3bX|zMm zi-vk7Dk0}m2yu|#mth&m@W`08MA#7~SdW*ca59J!q`mVX_?XeRUpTO)GsWdB)9NK8 zJFS-)YX-28{{=-ll?z6Jal681KWygQTV}{VIIrTob?(@uCAUNwfxx*F~terrK|4? z3JR<(EDE2`;u3o;5YqUay?GeAeTACUh3|E}yW}E5-ABGkNALj#v;HDxVHw45I?y4V z1|8rKZ6B9$xKCZFcLZge{vp>yWjm8ASCZoEtY}D-S7KTEAcp-Y?=eNS15bVXArC!} z-av1}eNU|UUHY5um7?KmJVELn@=*-nQiRddt@Z_akJH1E>UgdPyJY`jxFw=#yxx`- zYaZjAO5z>()`q61x{VU9BK&2YzGrK8^4E^Qzl#92fkax|c(YS|nRUn(dnt9)&IW8v z24)eH3jW3tYd=4n3YKTaG1piwD|J1Wv@}AoU33#p=7u9_Z@pRCqbc*;s-jnBzoOoq zpZ^=67WTza5K!)Dd+G4U->HW7o=z+HoIQ`!wo*QMjt1z_If#1;k*3G|-mJ|nL|#EL zh%6dBnKoEQHs`QH!;P=(Rpoi%d?2z>#;}eu;k#wAGA)^J{`7qW|Gsd|Jr*jw5Op|-ZQ)bL0vs5T--UPFjy18+Q z+?aQEA(k};Fc&&^)u9PdF|QYsNppkV!nwR$;cm1D!8O%@k>}#eIlOjeTpK0=SZr&a z-@r+J?176Q6p)=D5^tj6dOxZmRf<0<2R>IbImm|#7}$@?P<;dgQ;|;p$j23pQ4v)jdK3<<6a)U-Bx&iMt<|)ST^ zS=sZ&kQ&YJlZD4Fq5??5q2bpv>87I^?q2|kW1cifdtQ+hCp~S8CR6HKGQ4BFrhQ-h zjA*NLIO7Gua)7)W%kk&7)AU$+o9)E4{^nnA=Tkuzp$osw&m80*S%*&2Z#Qf7ABAlV zb2b1^>{qOk5;#+cS-;cEHuX0{YvC9FBM@g4Dj+er{T`sGW}yzCh-j?Uemz4I#SA5i z_*2}*042y}6&}KIgaGRo28)bI-y{`%5O+Zf^(TuwH0-_DZbPovFMCFWQ93-JcJ|Dj zGdqLy*nREE=eog-rA4%*0o`?krf^D!RSIV6?0%uP`6TOTW*0O}DQ&yvPv4#UAS|Sn z|DQWbHQbN*35pC=Rd;njg}|;j%-Sli9KXEz%GCYt&d*&5nOfrK=OIJCz!8m{(Z4a= zU%vl;d&@fHPH-)r6YY_*0s8AfMWub|lEr>YX{fNo>=~UBt#I7F9^hAf*>-XR5ZpjjYy>EFnK$%=P=oir_i6y2VU3UbtWVSf&|q%YAd{H)uY zq&*t``Ff=+iIn7@i(3fQ3(A(cu@A<#T;#ly7VSgrqZ4^=T7|ro} zH=-dvi|yxJ%q#UuSHk|H+ddX9OODpQ0*7v6haN!esA&8TpcdS?dQGqmYDU%1gI!Ek zF*BBXVTV0f2-|k`%lwgT3vIKw{kXk12ONo8!TE%MbO%dD7C@p!*$ouXa{Cma3_$gI z$vn<)lt)5p=Zzg*BZ`oo&G;Kz|i#{4zP|RD`uh0^U%_U|zE+EPB z!yS|jD!HMfxmeMDcblU_5r%g!C~io+Mq=g`bQ1VB0_b4{qsKAUXRv3|^-=<{FW4`* zJr<}|cpiT_8Ax%{T;x1p&zQ|c)okrSMIn@cV53%EO1}NqcH#$)4~k)r*!o}h90bB9 zHWyT_G$e{vDMq*6=VizZ>8-5hSEMZ8?_dtH-Nci#GL~F}(&L)Hd|&%Z_hmrRM0pgb z<3J8utnh90=KYW(+9TBHI3kBz#q#wT&zV3CjVOVBo~k?_TwQyMnXc^GgqBUhH}=e+ z!X+T0lmHUPtrPv?VAh+CnG}jGBe==e_oD}7P%5F& zx-1_->Y-FL^H7K2;GqiN%YENDBXJpbg4#f8{9)WNT{tGS=X=(M#Rdt>|LFL2sNL16aW7MdC=-B%VU4yNH?~%b#YtX;S%C~I ztZ7-~-I<5XtLpNCq3t39LR0zf)FqQphx8|aEhdI?-fX2$Gy1f37IpN-XRk5F=AP#o z#eW`|?45ri3Mh7M?eM+gL`nrgqdm=jw7{y)vgV|5)vV=k-ro0@}}l!q)m7rAgj(+>MsiqTiVBtj%!KE}H#YZZ@A>--e89&kY}d|{4vVjorx zf8}koSWi@PiHX#FRcjLbS#KTIW$$xbc<>?bnPv*^JxH(Zo7t)FT?QGxb_04Kp?>LQ zy=6fgwyof&yJXhi&^_t58cG^7e@1G-N%xiGnLI!}(@ykDss&7uHfaz4@XPRmh70yC zO&4-dMOc-bNaf68BnP89{w#@FMs|(;WJ0z*WNU#rtgdo7V&Kxjrue6nhX54V3q)c3 zW`1Kw_*pa=;|1F_ik|T#Pe)bq=R0ZU;%RK>6jDj?9~8LuQKl_*mgQ~F_2vwY;4DPb zoT0I0>7z{aQRSF_Yoeon@ahdVs6xB+J}7^DCN!E|NtV$kRxZ4Ps^~L5 zJuuc;G5FL935^7Iqz)3-?I=lQ;6fE6z5-N`T32R7lxOT>C*#Gev;f?i0e`l9Z{pn2 z6NDGIh>l*@kxkOzQL$2!)wk_y{OsV4B;*FG?_#T7>LB?=oh_UH@Puz(C3B(#)3KP))QvRa=HHx@)wVn)^7iCNi3in?LyXl#}kCkZD>g zl%)NR=@yq2@Bf0R$r&cF~_&po$+;t|~!2ep1E9r2} zfsMr%L!I7^t781AjzNj2`?98kWIp(Z#c@bi9V1U6Q)T6V^)!2C@<4|b1+`qC72aPH zhi40z_<%<~q4XfH+4`Y1*s@^rCsiy7XOoyJ@VQSPR@)IBF-=EG z4hp)*-<2g*-L-vsrU3Ni1?9O1(-+HaK6imep;qlCT;bEJ})7}lc87qQfBEy zW*IBGotjkE>OEr~fx0IkW^@FDC_{R;6CME#nL3RC@NRMt&wMQX^&gpPmw0h25TFXC z6+Lif4Y(7{1vR_{x0S4x!c)VTPZq@41J6#N>g_AgQ0S?yO z-+GejCgOh3?tMK#-GNk=;q(i@`jG!Dg^e{>l9H(k+Gn{wOO{lnhGRat z$j(6X8B*yiZr+TA>261g#**9=<0tgP%BKiQ_rwHnfDU0d-b4Ir!1Jj$9YDs01k2=- z-telTI|a6E7XDgr9eu}jC`8u<)YroAv~myfoIzA-L9;nTYJBB)BdcO`{BsXpk{Wi|}27CWmSG7gS6$ocLY0T^bkhjI|iV1s}1I(fokB| zXGhl8_nM^dLI;6C!Q-Y&UAq}*9N_sf*y5LfZ29Sclbc35ljrmditX3!uBw*SFBk|> zGEqhGa$$d9-rdt$=Yu#h5fRTK7!|mp@?qG4C|w*h=*?OW*GlbIfV8cl1;@i!G&tDjBLiRngbx zM0S@9%Guz5kW7X>T+L3}^W`C200>0Qw@&AUqmS|NM3jOxgCEE(%!0V zY_)zzK#*sr@0JANFAf#B{QEV#QvaCe8ldEWZ| zTXm~070A$Z_pyEUUVH6z!WHBsFwscRU|?V{r6j+CVPIf?!@$6+qaXoSV%yzMf&UO3 zMWmEbfG;l;<1pZNR69uxM;I7%+}Cf|iDjQh;3Bt^n7Wgat*Mi%frAN*wSkkJm93MN zh2c9F69-2NTN`#}HfA=acjit`cDyVs|Igou~_+ZF@n-hd1f=`30Ac8^_5r+T*E{aLPCE$pJj9p!D6dU`wS zPpts&MRuDHw)Fg>$?Xp=f^{Ri^~{0eM)NX&wGl|3n;JhF&GElcY#jHCxZ3V zq%lFzrUbAW%f>|_93|4THiUde^99!ppOHp@Pu`&uJ>=E3!){vh` zZri{`Hh<0z{FgiKD`zj7jgDMku;EwX2mC1w19~hHz=CVMYkYVVbuef2BbmbwpOR78NlFnTRYg}6f>e2}>Ph!>jPxGB>+Gq1r0#E)RY zt?b$Ss(oG$dUlxNcF6ZqJkXUo&#FIAB94%IKECh3dZe^2;(3vzcb?^tcbuMU zv;{#OVtDBI`?{}(wx3e24ouQ+=YUoiqnN(7tat8@%eu- zF9R2Q965gVFzH~+iy$LDzo*F>d-7B3HLM&%CcB#GWeDxEaBJEa4P5@lPZ}L4{sWIx)Xs>;%Vz+KxH#*Xf5SXF%hYWF3~_6hjP^1q0ZbtL zF+Y@z(q%0&C5H`W3H335hIdx!!KefC5*8cA_g8IJl|*i>pkIHPC&g|LhSW1dwzzp| ze21-39Bqg{DQl-8gS=q~?<$-Rs+TBuXK4%^JD)kvU9i1gv)bx&328%yW_8@NZ( z@I#2)Wi)0i2{P}aG?P{9{mnaHVH?D%T ztOmVU$nF29k5EpK!Uf);UNz4|PEP*yeXIzV$#liyk+PcF=7sD^=hhIlFe;U5qQ5_E z0@!N43Nca)DV8vOCQY45+{}!wqa+{!@Hr7OTCP7@OIw|yyMJRh%rApQ9W&dB+09&6dC=cm)T6-dit!IdMG+a9UU)FUuT_OsOU z{P@7Ij4c(zrMY!%SlxNahO1E7Qgv3+4>vFD(E6z7q%L;EwBPdks&(Hv^;JDwDyudNa<^hvgHWyM-kI8y>h&?DI{Cs-jQa^>?};lkk|~i*)OV1m``MBy~JItv;J995~XF^n&`$qGAS=4 zD=OM`%Ofi*>$SzGx1QlR(s^>@*oDc36O;Y;e0yErKAbO`4q{>15v9Dx(b3V_>`MT9 zZIZ6)Uu+M%wdy`Uj=p4kkchTB>mk~_xNCU6Kair0hba&^Np62M;j~!_n>{cA8hzaH zeclOZe-8hJO!F1iq$;g%W!ID=YrE^jvO~|o^1gc5(qJ_DGymxZDusJ#Bq@t|_dy)~ zUhw&4uHe0BIPvJi{4<4NKgRlPKgXUrTi2}5HQ~vsu4gZ#xJ9mP37qMg?sFPQdD@L9 zub`$Lvir zp$Fo{dfT1j=9BGSQrmy2W?97TE+;FkEsy7eHTCt?jyrN$p2s||Ehs1|cJ>hI4Z&mc zcbq(qoMc>QkaF4ln>Wjb*3~6ZKA(mzd0h;X33{b8H8r*9ty)Q9ONYUkA7Gthzj%jf z9Y1}>K7yDTne5es<3Qy|XEU|r+2Xfeo_!^52k=509zx46uwR~%N@6tMca7sSKNPkz z>$sqIjn9`J^!JgwE`2dF`mnXN<@J1#z2lEek#gQ5@4c^7qEQ`ke(rF6INt>+Z$(Oh zx$$hj@jP5=#4DKWPj0)U$?|=6Uvb~bJc9^bq>ip#+|9Ll^L_mIvE}9I#H=d>^Y+Q7 ztZ|z}*ZXRkKKtS)h5Lo6uGd))qA#`suzcOPgP24#6T}m#OeQo~ zK0XyOnA6b75esGa%nX&I_2QQ&!f8P|K9a}HE}43rHr;KJZP03#Y@c0}K2z|jchwC^ z6m;&m?qXc$JNiO?Z*TmKR=v$KeU~HN^i0-aV`4%=RQBxjbk9g8Utf$nKLrB=!|vfB zb-j|i>~Q}t()va3g%?i`k9@C@hm-c~a_=L^W+yWl7CS2|Ycmly`a~eEVfYcS?=6%t zLgE3~WmS5yUHi7a7LtIe0;L9PAc;t6bT*2%Q*`M78V;Do5;6s=$Z%a z%QrHB=d0}Il`#A@p#d|amITL8-qVoz;S;B7;zpeoaa4+w70@n6+QH+J%bmxg4c~Gn zw@ zp%#+7Qg5r5n@-i8=XQ6Cll?NP3d}}T;eseAi}vle;aehio8tjf|II+f8H6Pe)ELV$ zX<=c(+{T94iCq=Cx8ez%8q_?{iKk4a5~NB>>9RE0S%AUZI@Z7>+6qbCf3z_yWGa+V z4Tx7CHx82zqj7P1H_ggxW_f8lCn!ZtntsFlX4 zMks31Cb!MXC)HIhJ+aq;OX8s`vI$EpOauVS-MHhLj#lqPl*H&&2OnLXmKF94sM9nP z9QZ`=;Q^e@6q_~9J#HKVSRcAW`<2lR&jO|Hj|WsmtJk)_0zIt&i7EB`K>%eB%GHkc zkNc#C;xfhqVKV9O_%rL3iH)|)LBO)3C^WQ8Kgyh~RsRNmaPazQ(fxj}0DU=1ELP1A zEPbRZO%4EGRcUXRh)u0T@F*h`SGus-%&`=w_zP>A;kyj&iuKlv?e%4j+u0?P`8v^!ArFw9*1$e`5Kz}>OpHT=X zko|G{0#?-Md+rCo)ol#%I%A%Zs2-Z@jlLmsD7X8=?W0c@d5H9W_xB#Ju1ix(%l%)h z#H(NP7xrT;Z;7j`t2?_$OG~@G8}-HNh-7K|Tc*=|dgnDYHDx!e;|hxt)7N;nonH9+ z_d6UEyXU)|$wf$=#oqq@#%rttoUGMCO}MS^!{X-6b8-7UV_EyNhu~Sb{MwbrJkZKF z4Z6?SV2LZ?hoBc%7@}?WmzHuB>1CuPqzad#1%6wWuV;}3Y))8xZnI5QVvR2$4A{0# z2o{vl->p}v&abaWsWSR}G?6I-jq!1R{#fkhCJrumCam*09h&v9)W6u%svG^)+M3x% z=?g?5>W^j#ka63=glmb2?vul&`y_*Sc{nJK+JL?sdZlB4ayTO{-NKbRm^B?K8koxc__O z^SDXrxaxg#7R=)1&v)Dq2sj*rnQ}Ac^Sv*!wfY}c6aC%wifh{>Ag$jZt&H`nVK|JP zXPqF8QX;Qs`hv=#?_H+rbAG?lS<-fK=IB0_)in=qq91bkUT~ozz-hb8n`qnj5Lf|D z1lwy3UK@xub4M=6Z4Wkvm2XD{Zk8-&7mi%8CJHG&zqNXjk zbWS{o1hZqVuU&lZ2KCNwd|z%Fw3$Z)ZnS+~ZhW&(yiYtgZ(0tk1{OAdn{ zTL4Ic{+w12FOOXWpVClZ56^l<(T(m|io^H^*j_YPjJ1NZc}uINwH0+OK>#jWsTLb8 z?p-a4w;eV87eqYch%6^aNb282Znx4$R3n8g^vHVRx!UjYG*0>S=3u6j^rgMWFt@TY zc4TB^vaI!H#q%z*{T98>W_jTL>fmm${qdXNZ6`L+mHM%jm67jUD}Vp~Md4cYV)uQh z_x+}(Mg*9eTkFc1XNgw*ae_fhzt&*irv1bo2j}9(EXimA@)d4{27RqA{T7T30!9DD z^F^a-m|XyqP*UNB>(?*-*HtVs?heOMfj*-cf!?Q^ z)zP%ikhdfx@A1*TuWojS<6W#!_~Z_~&8eBOSi@RF&B1h>Y3>I|FH`3k+1b6kG(~ zfPNbQE)u2RN`^DwD z`^h{s^XEEYh)IiS%O`dAzgVKc5oj@vimvU}0Fd13!)C+%WR?mUZHl+eqN3rl8{H6hdeq)ApdGSI=`B_)egcV zW+J3P#Whoi>NjB2;pBVJjE!u^R}WM9-7?iLM#^woda-YznRo9t?Cc}xSsH39=pBSU zt0rle7BaYHXlM5$A_A?jsS=@P<`-ZEni_z2G#VYN% z#*SDb{i*WLRd0KJpI=CxRNS6bCoZ*v!zjyzf+0^l0ASz-5Q9W$B>&xNb4ruVoFR6J zFiE_wxNz%F`wSO!6PoLQI#GJDQaxOn?sHlI5g|^olN8MCFhrnXeHJ7Vjio+Q&L5L~ z^UCm`Rtketnc#D&YZ0@22MRBm$4KtQX%t16nRB83!oB$Jf)_9b_8)+8rr`aBvr7y#|i#Pa6S`tdSvg&fQ z7G#iQM@qgI1yLmq_ng0CHAEcQeB$rW-&aKW&nr#dC!-afs*}G2~ z&_4g^JYx!Jrf;u(f)V&>&orR?hJ}{1IGCYaNtzZ5?Wz$k65tn`z==?ekOd4RQ)RP)WANobO#KJTH0;-$;`&XrIrPKhTuuM6d z1MI>~^O|K2yqgsLRoKjZN6=&$EN6_jHi{Qs zg95z@winu5er+^8lmgBMp{+TmTqF!`pZNP`g~o4?^Uy|~s<|hAz|op*(6)fWBHKkJ zkAk$~fuj^{wz|u?#<>ryojSxDUzq;togF#rCt)4rrSf5iE20E2EL3?pXvdBb3Km8k ztJNjQ-5KE=wZ`Xuv-j^Yw>zY4~FBoMoxBFC_hR?qJncKHcecNw- z;TB@Vw5{|o_Jvw7u^701ll8GC$n!fV-Y7<1pnGgLI;*uZSCr5>;!+m49x-T`ecT?~B+`6*>ZsDSoL1z9h5@-zEPX|vvP*0iHMAsg(z zNR1+zK;g4TASHIPKJnF z+p)Ungf((%CcJJ*hvZ}bc-gpit-ZNezf4BWZOX zSrg)#72FM)%TTC?XEG8BsC@H#$?M!PnKC4%)12-ra<0$L<;`$)DOP&$sk*FS`Jl^o z9hxEn0NPp+tW>2-)uee?rLbaTcz52c;Tg?)5gCM>H+>TIB%+5(C(IjOGUQmpF0y?6 zGKIVZxb^qp9GsDVEUIw;SB2eB(Dp9fKQN|~uRWlY5<#AT`cFLNg~5`mbM9RimXN|pabqUo%*s*`{BUAV1y$g!Sf7X1lFZo^&^h*tof)j3 z@J6-FJY?oEN_@FWn(;kuveOFdFOuS=aFUnY&(*g~=j=aR9)%{qU|QCR_O)r4+-@yI z%yh>QhkTF{3BT+IX5^Rhr(?Wo$SiZ~|0vHe_JvjAf3_5Se#@o?I}26usrGo)msVK2 zUOkr)E8bi+k26bIOuPu~_Y&NX=)V`>wn*OzS{|oMgy;c~0a@3|Y?ZG|yCydy=GgKF zlb|9;0+MHBg;^v}?XP8=e-Pm9!ia490GdQEU+7E9XleT8;ni_->iD)^>Iq%`9 z0>&a_Oi&j2O-e>_I+NwqMhvIyn3>(|`!mQhTcFpoPW77|?qEOyFRo$qy@=^LB++bw zLZQ>e?oNOAH}OpIvO&n^c5FDZ@NZ<26_bF~Q$eR@tf^n5SGSr#klVEKc3GY|o5{N_6Q=nXwjTr69e-PTnncwmV7C)#y zWB{~_vGDIxCh;M5#5` zxj)@?z^aCQ2GxIo7)uC*g-^&pOd^EJEgKr`f-WfoYLipIV&4pwp$cHw&}$KoZF0v% zu(Ai)WsI~RZqIAzbsVFVRyelTua15~F=0Qw9_`dY`cA~GBU-{a=kUFMo9|jiH(A+J z=Q0!zfAhUJ6>S_3h-y2B1>elteXk#xSC`FVYe(BQpcjnBk!Dzh>}$FG)1h!@GT}?n zIj>Zm8F6?%D1DeVQiSOf0Fj2k)UmX2e=9hh@w1vnF^+;g^fx_o4UG&)R5THF%2Uwr zAS}F_;-S+<;kZxhK?oqlsaIA+V;`O7H%j|(lju#F@DDj*1|)&<^q^o7w*%DQ3uxwe zOrsU!2JVIMk>JaI{sfQ%5D8PD&k6y5Hj#%-FJb{CGolb>!BVbr0* zO~)wxmo*!t8`neE&4dx9qJPS16WCUYbNty+A>Os;V~{p6b?N%niO4U?Dh<36+J3~} zz@RZzvS@66IUj|y~&ZciUSAAoy`U5$8H5#Z1bA5U- zK+y##lcWR#PlH2)j2iICB0zi;ti8_7%<$-ix57 zF}3Dfetlw|)S?%c?p%nZVuOLVm3Nm;oRwltdY={d6;&|TA}$D5|CC1HC=ZaCGC~m*M%f%Rgs3a(v4BH2_x#Fk}%)#%Dmg z{;}_zE9>n`8&NU_sXs4TvM>6!T))UC91_xkWe?>XtJLhjmW~#)M7R-5g zF7*^<5KIq|KMS+ym!1G#q|J1}tZ+Zv`|E+o?sV zv+;R62x`-H*{0s?Vhx~0!|_Y3?{@y5hn2QwFCwRSGFFwN!D9lrj82o=1;=D=I~uzl z!6UJn0NRo!t2;YdZDl6S9DMp5K|qyFF{S)~Dw8Ffc}zxHfp?ofbI<<{&da5UZd`Nq zXYC5Kc11-+B~hX}NL_npj!y~-ZYfOz*{fG-;sg2dyLpu_H(6vNc%uUM zzuE+_tn@&KZHMLl{+d(blVnX2F{;DcBb6=5m<1)ZdcjS7>oA2*c|a538{W7XPw>RR zWGnQF0SJ1zwDuTwArY+AJJO8CU7_$Dn31oczm>YId}pERC8_|2?o*$+$gJ1S#ChUy@77wnKBl$wX8se7K^UpYQ?AJ5$v*PMqI6Qakk5~f)nfalsCK11p~?I3w%)Jr z6a9ARCyVWRxrUz14(#55#c`>=AdsAZZ2*{iCS$m#6NgK8Ofu6K5z~|2NYLs~HypHFtPk>?|*+^lCHZ9miRvt?aEfe0Ik6!mOXph51jMZ{Hxh*1=U* zm)%F|33(HnOismwx>uas=hRMZ0nn0ghVu9oMS6GYh)`t!DpbYd<*VvgqXQ``5S$KY z2yTBG)`}kSu%2>uO8_`*7h6gURgvGgON5H{z#Y#D%c;lIV{u_#xmAXfE+g_jbL{%< z<7~Er4O1ZFwwcp%ABxovZCXV)#?w&|s%@&wpyxn^=^=CQ8+nu8XQdqgm99^BN$!_x z#67rgpD(d~=d0mEu-Fsv>MX*MRsv)OFiyT(e^K0YBxq2K{C%VdvW1HzaOIU;Sf~3X zovhd6Qu+QhD&(!GDix2cN}iaL6^khkp%7xj~h%SK6AwUzk5|1&#ae2snQW6F?AcW$6a+l z#154G?{)(V0(E?5@|~@w^j!yQrq}y994iO5>YVG$e4P_Bn{XXcBQqsYof;WmgFpeV;8DYR#1P9l&pRJM`5(6!R(oZN_wI z9pa1*+6b5(c?*lxtL^58smQ=3sJ8;f zW_6#i-r6Y{rz~E+a95B~6yXrEbRgx_VpG(Nt_Xvh)4f0K*`dsQGo9^a?nGc!8H-## zDPB<$Y|deL-G`e8Lmeq!A{Sm!sNHrgQZ4K(qX&xaGR-aN)@`UGfQ8~irw_ufzpnSU z2gf;7|88`Tks2m4Lc@-@aX(eUG65NqTB}wP)vJ2JT zl70$yHmTRC{Q5fmg|ex_jL^zGm4hC$R4a0#9qgY-q{jR?0E+N#BphfMG zHrU-Ep}gnkPXy~kT5hp>nPCxMqug{_Ra8_3!>hez>}Mo)EQHLt6APy{cHx z3CetNMw>M{j;rg*io&r{+~VL%U~R@G(d3`VA2)1sb}--aBBiW3jk5Rcl7Cm)ugI(2 z26Ex?8FD7OK=krbzP}&JUE?K9?09urj28crmOrNj(lwzkb5u$LGr_1QU#!v2J^{-~ zH>bUM;C@|qz2Ul82?_>BXttxbFOICJ(#JK`*CwBXkp=5B&Dlg7HmszrPVc{s5;j>F zCjMmcl2?)nr|kq#r6p>YLUaPQK4te888-t40oUC$YO~$C|gwO{Y`! zCx%Y|51@IGHz3U+E&WPJz%#$kDQRiSgNsqm^X3|eLU`LUBA{r9}X z8w~Q`U4`6qkZOb1#881Ie%=L-1pU>Q96$+H1>PTE7Yqm28-yTh0BDC|827#W7O9NJ z&#yQlI{6Z{7fEMV{4VKAUU)aWfzm$J7!9?$8Cj!Q0Ui}QTtmgHnlzZibl#e)S-dl5usQR(jG2|?(Jo8(r)MKX}!w)qb_qAm8}%JN2IB-tPUdr1Ez5 zxe6r+wl_p9@Mw)0S^m2QOs3QTaC0XRQ5eAY!Dl(Z}oR)*@O()64IXMXy)%KPt5`5%#3{4PNG? z-K6x4>>)G^4*gl4`(b4YA_N3`m28+4Ax;hp>jK=3&;5q6cLiS4$kK07=)x1Z(t}W3 zJMhtYH%{xcoA@OY*|N2Lz$=kwRb^v15wW8Sg4LhAv?c)j{_5Z^1Qjh}^1k!?o2#TYbBFN=P_?1Te3} zv^EE2Os0|FEf`3hB%fN{f$B$v#?)Symnz>>`=GIcI1<_Foca(XRez8d& zU*2RFo5tA9Hj4y~#vD`U!x=qRmTeV?PfzzA7=#$%eneuYlJ&#@JAFLplZDnz#<4zG z=`f4>C)ZjZB0`hxTtLZ_*pm7+@wopN2H8`br?_|UIEmyU3u$qflgz(%8-lkFYg-ER zz*P1*3V$$t#?)<8%P8bFe+-D+ufdi)3Lu_ghyN3o@-@_0>wE#SUH)@%u5e|fAc+Ois}fc91Vh;lKzeFs zdvFQ;(VYqi)1W2YUxNFjK(W0ib=h9>j-FyY0r^A(*r^+>RcZ+-fHdFG{lEj!O78ax z=Mg>NUR#(MMPm_k>&c6I0%lG4T9sq|rn}2qCnV0H_DoW<&&QuwSP%BM=HXJYFR4|c zGH?<$%^Rl1@_;J6Xg-Yj{v8sKt%fkCEZ7VD{IpTX=T(0D6ZSOKFdpmc#!4M;I3(l^&6{j5$g+$#m~R~Ym4zNl_d2ctQ(1?v^FYqW zzgegS@vfPA08%r+j{t@0S5aL5MtUd^B>~kSMf)Na2&^&uk*GBDS-{$({w|H&*G6iQ zK8cHX18;D0OtCr;LLn*+zfO&np2)XS9Zvq;pT>M?e@xp9e1-f6we9B7zQk-{zx5Z{ zzKie;WXw&lVKAvI*Kj@#Fd_*ly z%_nL<=bQ|KwsN->SM!a@7eLYHv5N_;OYoYhN1t_X?-SB-x6Mo4o4I5FRd1mZ5(eR5 zz9lxWUhl&%FX&a zKyjP|>t^@|Lglq#7}g*8lYr2dxyz8zQ{)i!PW)#~HPkaVWze)5DbR9>@B2|W(?ua1hlnr!>3OZ!~oiZ!nE=vHv zaJW2TLZ?P5`$!YP^7PP!1n@&5I9qp@=$&e;tXjIBrAz%?%4@Iu?E*UO;)dB;>Zv)^ zc0LUeSs}aLsiXdS6NWJ~hfY{L8wD#(Z-Ur{r44TdZv&Q=En{^2Kip>ogjk_*O)hvM z*zAlAO{(+w2;nE@-H-|{Bn`&jkLew5T>kx{(mg{!2Z zr)JDg7;Wf7>@X<3QiOp{0kX0Vf7BgqpoCdF(}6Bs2p~Q=$TFrAym_gpbRlPs&qS@r z5EF55bh3WeR>f2OdDFk6HXI1x8~Z`fIqjqM9@mgC+Q|g&e7Du~n-m^8*4jI{7V`m> zqc&#?QpTiUuaMh)|L~`i=*m7*=V)*dDmV7+m9l(ZLX#|gtvr2gdAlB1p@*8BMT8Uc zfO+uXva@nspnpY6Gu>=pJ6&CplOkx~yF%qW6ETiqoY!+rh9x$EZB&+On zba=2aJH6W;pPe~N#*0vgrCw8SL|*cL(kE`=B>>)eMuP{UQRRfjG+*3~WeJLBm^|fS zLi<8)_<>dXl>i285&$UD9PS}jZDhsdBMa7?9Q@D4J+?XOs^_Q*EU@W`;Dbak)|&WQ ztke!aoR=X}`pNruziWY$HuP({IkJ$aT14wcb@EnwPhRo3?Q-lPT3hCUA} z3lIhH7pFo>DfGe?OJ6}|S%$Q2^BRFi({E&&ql8Pm0BDDwt=)6( z`@&7;jwU_QvWoZrPu~C*06>^G{Eks|$7{G8>N2mX;d~c-w-Pa#Dtf0nLwkAf%$O_8 z{X)(!k%4zWewz39A48G^(vT{N3s=j1E3vQCtB(1kE`GpRQM$6O1>rS^n^~c;Z1_9B zxbUrZRw}1^M1cb+NROUC>`y=aUO{sq!XS@llLTshAhyPct$v)GVbyi_$GCm6aK(*^ z2|g0w3_jPy%Fi9b6b5DG3$E!#s79g!i2$HM`|Lx+8JTCq zowhprUJa?g9!Z)JV685kli4JTlYsh^wio8u11+dnN+^VwF$rF~wl)nfQ1MU|9nixA zN+IB2$8k~`jd@LqIgpD17XX1r>8O+|tK6!&*LJzs1%;9@Ct=XJ>~`hKQ^k5jk8KS| z8mM#F(?bGwaombJ4bv+P-Q^h4w=$4~0N1*Um1fWe5S>7>jyR^hvr{)XyXm^vpLfb_ z4a6VQXH6NVK>TBW0B>PpJzxp*HgbF?5)(_BkKA}e&ZaV;B?!0o7J*M@)MDqOnapRo z{OUtbvlJcKt;g$;5E`qB6gkICU;cMb_q*>#$%%j2asV1$=WTSlq%Ux9`QdDzaj*oi z7R%WA=p$7`fmOlM?CQ);yDLEVS+z1xF5u#m2%}}!Zpbx!LkilqlrZvBlZ!fi9RE_r z=jxETN33iWbfwlYUNB5Z4q?$w;EdP+2uA$S5jB|@i-X!~G`Yr=$)*TgsW1>25YJ>n z;O}noXtg}w1U0Q+A?IF6{D=bN-YO#7IbYX%#MgF}j6Tngq)tO8JT)m?IFRh-OZk1B z?5>Fz5ls$qcbs=UFr^1f-kdN1hxPE2X7Zm&Qi^7(Q;dGBOrFOch>;(~jB$}Nrg zEEwc~&JX}fD0PoU2k~`5_H6yiu64a3%LZ`|`N7c^55-2YmSeWxE2aU;eS%8vA|>ho z4VR<~fWS-9-*{ty`e6d~;ua%*1u$(}rzoI0c>B+jlUX*sCtz;;t-dV;+xIzMwS}4Q z3if6waio*_EC0>tHlUMugbWARWPVO3ZyF?QKQrtH2uY?^eZH^vLo7+>n5Mc^F%IIu zJG)}3`)djlo=%H(z?ELaJe)Y&lD;Wf`efOINxsmF@JKfaKB_SkwHI3ueW02zOe_xu zX+$0!!2i|Mgvj8E%Lz|wm|CO%QyBrNRr1G4#*h5D{!CG|z|TzZAt@9%@KjO^S~vF% z8i{?Y1!>)k*-^1Dbii&l`>}_{r{6=2mf@HtIWL0tv1B|z^Uxl~)q8UIG zc3!N~eRfVhdR?wr{-1YJ5m#CtWU-gGl7UwW1Gs%+mk-5Oz+gM0&Sxxvqll01WDsu; zLlwOntT9sLVx(x&*Sxmy*!u|n2s)gYSo z#X-VK)k)c8<7!+2lG(_Dg6T(s#Z^fo_ksAWL2)B>ShvtrSxFU;BpCLBNMR=7e z-i`e+y6^`}(ksHY|2M;{afm|<$ALla?_X~!;Qln&1y6PqbGlMGeQ1txx?(!NX~C1O zzDJfr_jV`rVZr_I%ErgjOn~!%$&5ly>dQ?kkDPBSedg7DpaR&W?&R%J0z!m|_;ybB z2B5O4B1WM=PKPHMMRx5e*+IN^MRdQrW#uDzIyBGFfudIOPu=UZ_`klo;7dqpS z#Fdf|aE#SK;EQ&0$KNoxzU3Q?H=0T%ECmE!pgv2e%M07*YXX-rh^?a4v5$}}P(H!Y z*wbKp6Z{s9uF**eQ7wap0A97~W34XI2UH5r*2ou|12M?CUN^c@K@L1163Xwn%<))( zu`QL#|7)b!0OAMc!2pqO2)lj8G_R@Y&KP!%e};F9aI?vpP&`kn${g!P!mZdr1;?HGaE}$a+^)Ev4lD0*KP6$k76awgz zP$T7WEZ+7DCzrBKfa@xvEx$2ap|&7i#>NLz*CqD3#RYWp1qnmHjh0@Ra3@v5%`!Oo z!xIxoW=!-tNaFqvM^f^dIRZEcq8QA$eoN_0&blc#`}tCC zD?*K^X0+GyhZoJK+}>p5VXrn{*fpR)&2y73Czm5hBcsrC=$i)C8ujhMkYZ$1KcZvY zAH>mD1d#Ri@poidY>S#njLqpIm}2-9pFaq{_qGwac}9&VdUSwX)ZPmeF{7Znzg9xR zVrgZr0E`X`poluv(SR79M1mc-P7Nfd;k4}hzQ#D&gT13zm(eVr{z7~hl7&AlQ%aW# zaXLbO>TEyEDWcI3;E=NRncSA=OqH;kCcV;dmu=n;Xn^3F%rJ#Bn`H8Fj)6tQ`g^W)yP<{E&X=#sbVYSEJIg;&}~P}?zEud8SOXu;(ATZ z{y%S8YLSeEZ@hOzCo40$(uah3{T#Meyp=&&N>MZ0ViWp z%OkK--$WvMLH&yqkG`K`l_uo~ni|O{QblIokDJA?RxPMb9k)Xp1Sa>=c|{a5r}P$0hpqw4+B(1relBt@$BRI*c? zSO|Phxuh@>ot;I|b|xZo3(EbK=z1U8|8+HYh{Wjo$FEYS#W<8SR^>S#j`LX#dUSF1 zU>f)eH zU(ocOj#7SE{hs)BzL1O=gShv?u#Ju=yI*^3XQ%E8S`uyq-zhI-ejim)buTE_CbuwhF-!#`3Fb!MX++a{Pu;XdG7z@ z5HG&h$NX|-RWg^iCxuE66DqCH$-N+I;UK0iQw2^~M2?&{uEMZ=#B_llcB3512X}Xe zcDXO^xc8?Goc_EEvojb|WMP$2c!Pv9hVTW$A_>Od$wR83xwxt6daKnpV_$CEseo}0 zn&i`#E$<6j2K@3u9N1Hw?Y4F1AXK^n+C3O_BP)7)7}o(LBg) zNopyu9D7B<3%!^{9J7kaqe#$3e<1yNRGN)szUwSu%YF3uWObn1&Q#o7zzjWCpnWY^ z%>ySHJNrXSKh)mzCzCDenhd$V-k(1fJQZXj(k)RWLgf(WxNx$;vgC(PS*$h%fmUr{ z#)f3|mONaI%tq9SJtr=4{RC1w=uy;c+})kR{NGniWbt`}TIAmQ5H$v@7db}z$R0?S zvloh{?%yjjC2q!ys`s}qy}%{S={ZC|W7s=?^po=bp@(e=nXEgZgv(@Y`=!h4 zobqO^Qf6Pfj3%0b(L75Z%AW|k#cc=>~ug0&ZE); zO%O1}-PBdta?5p+_Zl?*B}9yuudIR)8x4*c{HxUZ+uVh}Ec@fz#^31i$0T__Ztm0K z69qeoqGt`q-_8p>eCzV;M1J=eS2h(trsYO9=);^^@qzdZ(Vu4^}}k zsQn1q2DOgUhNJP@%7xq?ANDM%J_@24uK)k|`UMT5Al=<;Km??_OQaj5?&SObbIyOpxc83X7>cYl*SluC`Ml5i(bF#=xqF!2 zb5$YuN{8n0)}DQS;1Mop#%1B)J$(D zP|F05EzrzIQubXoQ`=mlG@tXv8jdy~Nt!$($2Gb)MXOWw@o&q8lfYl1zR;ZNu+b-n z7_dG*#r<2ivgCNj1Th0q^<|CL{)jic;`n=68S0+M;;h@L$t z`x0sB+Amh}x+PsJp|zjny>GjXj`^63G2M{zUI`%ag+If@ z9hNFy^KKI}ODCpvp32XSp|QDQpd?N|H$m3C;<_4j_(Y@rx0%!+Ipy-EQdH-MXzq_G zg}WQMSoY5^w62~=7vSN)9PEW`zl~t;-1h!^M*FudU4htuSXo3w0neGmEZ?f}`Mirz zJ6k)EPi~Yzkzcv~K-fhLo})4Sh0)VJ55_km_;MGBG0p+D8-#>&!vg{)k3RmGe7frd z*S{)q8tUx0

h{XlR_S{wmQvLs{(bBd#I-=(|UlsJjL{G_;7#4Scr@5*8u@-zI_x z(4{Mw!lXNBTu-cZWcU=q>097f**0hy|szU zHQue7w*;uh;f;M=QW_uYSj%>K`SWes^g4Ya8~#y~ogjs48t&@0kWr?Eu{LV(^b&+o zDH7_|t_=J5a^c|3)e}mZvfEgq(?$l&7dic`B6ZRX8gvF+-75T^*5RDdR%M-E5Imq)2|5Noe>PvU~7l-WIB>9!v!c2n6#JZ5p%RGC*Ao$#Ya%MR&! zLLlpF==CW3)emhyNin9`?a8<8nb8oE(RU3*46EiUVQ*oXL3 zy)i4*et0d=wGZZr(H3x5$SZLvPt0)K^2!7p??|qYJpQfA?;1jyel@Wle9t#`SP1$y_+VN+n zfe^B>4$#~jJ!8?Dd$yU7o5HAlzGhrVZXu{g7|mEl=1nOtJ-y~3>>;GuT zpi@u=SDC9QygLf*qdq)lHOR-|<3L~Gf(Y}fj-9*|7UmrrbKtD$;%8@xgzm)?Prmm{ z!Gctp6je&Ywmu5D>*p72eROxX&gUWCl5V>E&oOBkzm-hjW(93Gd`-zn-UfeVWTX}j z{%z*kO4IC&liS=AGr%x9w&rYK92n){k|^5vv&4KEk4B4KaJE z8{`;^z^(Pv#(~?Un211&267SpsxCx`4U);qW-z`bE7c4+)*-n1yxn|bWp-(b)!n{5qy9BYKA${?Xmm&+J#X|~`^h^X5K07{GzE15QAeE(!Z`CIhrX3s; zuUTy~p&xN<4Yse06VOGhBDAp5&^((@n<}S=mC#(np_@|6bcXREX+asu_L(S+0~tFF z{J#g`U*do4d#6k^n0-S*ZAP57if|htiAS|Ye^F}xMjf~)_#_$Cm&VKJuwGGw#hNVf ziAN@HGNGF;uwJ`eytEfYS69&5icwXK?cVaf;Q>AZE!<{`%Zth$Ftxnj9mY&Cim;}2 z&YKIW-%yMjMQkHqOOk4;Y`7LFO1(p5L~auw9(E<3jBoc^CWey;v!j2;x?B>gu>6`q zmWt^00umkGmo!?bl}@%rj;?9-vgc($Z`s!?hvG-)BFz70o*R3~dIhJ@hpQMm?5d+7sjC6g1*$_p> z(H1dx-zRP5ugJvV$A*_*hq5hS!@rchW(qBry9xkeS>sr{_6EYO(4>Q2*bKU);YAz-vNqWnoI*!@*Tk z7)1ex;{SVU|Km0%2`THxv1Mw4xGG+zXux%bGO`MhIhg zUzs!=Go!MQKI<!Qq9cSxyR`?Diw-E#>e^S~N9&YBeib0B%iuNWG7aFBOSN27kmF^V#!R zKb5oHXrHDZ88+>Tlnxtm%SI+Y<=3zxplwiicO>+g$xl2V17u=_3nYj=EJTQZQI5H1 z`d#g7@0m3^XB7lZwB6Kkdk?piu41HpF1Qrj+vnQXDzRnIED}*6K8YLgXxkZ|0NwdJ zJC?5Sz}k0dMZ$z4c}OK`6M9y2*#7(F{?~K27f;mP)`_hzvANhAq zvAdEK1aizna1<7zDkXxFhBGfXIDk@4IBjTlA8lBd_56q_23_Q>z@$fc6Iwwf1)r=` zQfMFM@;#EbRm8K5=9QH+Jd-y_ml%gGkjMD3oE2qGx3(#K4^3e_$(Y;;DNn)1*H0JI zvUn|}?F*Y`OK3PWT4scQIo@X_K76_x;DAt%=9&!s_`@SR0@_smeTNbit-OiOTd%(( z5~{Tt+O3>$+%V;4Ztt=^jQ1k@M0y2)xfxX9nbMch=@u5#J#lAjyEx zke%%SMIaK|Mi`5K$mg7StDA(x0!uI#ORUaXMA+qlH)}T9dI&nx52Kj;^fLMBOwDWN z_HUb)Mb`8ZNR%LuCnpp4yOR>dUKj5W-X*MT_Ny=6cj--)7gjkcpc84nyIhMFotFe2 zLXl*&=-nlv$(!r-jYPU{h1So7ibq8WFp%ysZeY-ljzv}xP#xbA1VAIjCFs1%m-f$1 zs*sl(Ow5UnM`s8uKQh?S-SO7iU+k_seusW+yykmQxEh6Y;#TOBA$@;F9}OhG?7@Go z%1rh*Z;s?7k`_;u88&XrX+oR{WnhPBnBZ+0BQEj6f!m>xIO(kl_EkV_Ef0Cr4|I+PGoY>J?(Y65P@-77CQM`aW6q{O zG)VVFFW~kN78WLV8h5OMHzzTClg?>khu5Yf?}q394!fML&i1cKUoj7cjr++LjHRRz z#e^(7H$Bj@%zP>!zx37?(A!kIjwz%$O=>x~dnzd7}FFK_8y!Y=1g-0A*N%R>*V^5FN1$AL+ zsEeohycFkWsqP~Cd~9*196&_;^M*(6X+pgCj7dN29Gfy-i^N96;)nYeW9bHt9Qq4Y z7aH>uQo@Kx*@UdbxxaiKc|ubbpIw1np%yh^g{Ual7`eeF(kVhLZDK&}HQcZFw-=@s zsIO+- zNOjqc-uJNzp^SfD^lnYm{wK5-T%}oOO(6=nX|`@|lgEhE{I|m{!D4lDy6GnU8d;}F z`cdVj^`)(G*(uKfwN~U3(qoQZq=w_VJadZ;=j61$I4QV({Q)ZihNtM3{DbHcKAgE$ zvXe|{zjPiG>=3P^V)v$Q(VYb_e6Rz~pTSh#lf*L5z>{Iod+xafyIu-FG=oz%H30bB z`W1K=y45xGddj@cSRU9xj!rV~)6RCL900qP{WdK5c-bFMakt58j)&6w#_oK7VSu?H zt&=udsI#PQQO?4Gr6AKwpl-pL?1AFVz~QG!6o1geE`Z4=9|O9ykOB6#w1p;@v6&gI zqiaF(SwC}*R31ls`Hg8RCBj3#P}ngD=XtDtBed^SYEBR>U-|NJ4(1mpR7TP{xayVN z)t@$!N6K1r5oHH-G88vGyp8?V~%f;rejG5KJ zf-Ps0I)IGpicpb%;|X9EF}R+cay=c9ovHdX+})aa_45;8c{0BJduBOZ?pxP-Rzm*y zbOcZ`y=Xp^$Z%Z^WsbhOyJ*Y&Fp#uj%U8c1C$Cxc5vT2T&t~g}JmO)!9-`l_g0gk0 zyX$9_R}iDjaCt?&yQ2N(DJK*+qkEF;9016={!UV&GNEoZkpGEJnl+Jj*Fk|b9A;m9 z(cpRye;Pb}a#*5icX;-$RQIZAzrkKUnbNZ1>h>;l4^VROZzQNPOrrXW1Nx?!Ivc9* z?@Exea&t|LTZXqh22bwp?wF(90LRNoq>ZNuWF|5)(r&rkx68vnSe#G|8S_dv&C8MH zV5#2o2a2Pk4t_E{I6+e=;D~{#1`#kjBL1;JXu)1a=oJ&!Y}3Uu%4S*fG3G!Ln_%&6 zGMf&3fSoGso;~&z2N##r1JmTH`_)GBRh8&9Cm;h9Q53y(WF1(|i&spU${^WWwGjI} z!i7s9kkYJL>LN%{Y>jn%(@xTp<9x!*9bc1fRGZe4Qa&tgz^X94WjrwReGB&McjHd! zgOXfXF}ho!y7QF>^fcw?_si&_eGg1P@k%0qFL4EIMj;zOtH`LWB>Oz}6*Z)~ZjQ5R z$#pFfaQFcVU4BAv=(83z!{`+#btg}kM5A;_*BgBKJY?gFEpb>49pxqpfYL-Q^t^o- zS!e0*J21Hom#g7doy4uSnp18=f(Jr(cL(vkqT;r#ZuID8=#SjKi69m@F`X)N-_l}t_9Z38DAO_UGWC7Z4UC44dM60 z15&^Pe7nc^SGR8>7CdZiyD?0#``=_}KEEy^093X`v%m_(w0;rRd{58z?dV!yRzg3g zP)MZCVR3&)S+0;a`vUxa(G)Y8z8Y2dsa0Do=g$cnkt8b7#V!xU?a}LOkOYv{QO3z* zPx|;eo_K)f=>ccPPuz$>p`0oeIW~~1z=3EmReE!ERUWQ9<-eRKLX7V5t=JWk1{z&y zXR7Rh$PrBMiA2*SY4@FLP!7~lrukPJVdH_{wEm+qpPJOYNCtMDMns=UpI{o0`cPje zQvXguLegHzisXwxtmjCubT#Di2+edFLO{~&4&_x;dY{i^R_cnB40@FMALYL;owo-5nYV$rHupE! z3GRR^4a&m<2eg@jwKhvb8m-5PZv2YEzfjb--~~C9Aq8ewjhe!jEMDFSL4?e|XK>Lo z{C)Mxy_0Y2510=FJIE?@)TXDh?NPf--S-&{REua%531Y7VL89X&r;akPOYPwmF`mMb?XlJ4Yx8pUMwsu5YCkV@4?{iI2qvvu#tXPgGX|4bB~OSbQFl3F(Uw$ ztYiZ({*sasDF}h0@R=GK+U{Euu=m}z1eaa|4cbfT!8&E&ecNr}4JgAlk*O{bH1yr* zvz0@D@~?_n%QGi7NS~{9@F^eeHxyFnKFAlY%0%Ah!98t}knhGBWT9EaL%YVaaHPtU z?U;M-a`9H5DVBcKMdN6Jsir;S9=1@$5&vmF$nx>aTn-W44?Gwb-cO^vd$TWaoZC~T z6lX-@TRbl80lArNCkCVK%!lNos?6I6K(Uo<<8_kypvv`{oL8LN{vMpgXWr0d-mEh= ztcJ0WkddW+!CgZay+o|gi~vKd9=W<6_cI<{?e0t@>p4FI`=+3-F8Nc2TM*WZdY{~A zkrcp2h36x9&{MhR+d{L(%3q8cT=5i@oLi<4OR4p_i+~VnJQziaQ!V*5aR|u zv1ghbxKA~ONI@Q4p$En~z-AUz2aJiXZ3ik~X)A3x*H$Tf5-oJxXL0V-pu4-+l8TIu zi;s=QzgIoRkc5YWQ{i!84b<&fnYS)DUU@z zzN%_A!0~-MRbMz#*fe~++hEwEbqzz zx5(742Ov&8eBiw!_r5tQ)^AB+|GX{Vyhd@b9L7@mKoX|wz6}E;VK3Tl>|6g%l#R~K z$%E~ht!%o={g0)zZthndC>S$4z7kU0x^Zb&AN6xbE(hLy4wM0F9#>bl)ff5^!O(%;5e^HMLCxG%xy1ATRd&j>m@MPN80{!i|(|z)b3`$dTaNzV>@WdD&^ZM za+9(#Q#KnDEWg^BGcVU!<)!rXlH;ji;I2@LP+$`@2<$jT{=jQM|5m?OJ2&0tE&AY2 zxGr5U%0gn>H|&pK?4Luy&OPd*jkdj5aB%=ey2^6qWl(T%XzvAsI`8=xz>SYWj-NGJ>XR?f2T$T3FCFi>;Kl|J%dA>>$>-^4K4Bg z_I%Q_!r&ZnKSwBwEWgpgh!T(3-LZD-J&W)E8w;=&wU(q~y^>;4YP8h)In8nO^#he$ z8FLK6bgdOx=HKyxR}AsRx(#uFOpc;#O5WRBR720{DTt;}{KX%5?;ds&f_2j#&jX`l z5}S^UgTp^8g9E6MVG%gTklQWbggE>McL>^Cr6xTk6ul$G|<{Fv0pk5>Fia-PZ#V>#`NP zX-3dcRDI_GseCex$b~3Pa3yb;{t!oh-r5;tmHF!F`@UAJ`@qk)VE0e+`P!#NyX{bp zOLFYJpz}3QAD3^3Jq(-j_HB9=bdz#9CTju>t~9;otP%3iH;@ODxZc2dt-$&d=uE3`;3R`^5}bv@ABir zx@L3gX|ta5*}0~hbDS0*UVNH_?J)v+HOUkA7Hk|Gvp8*PI0>L>zdqli#pi3*YH}){ z7T6-$>)61Yb*YO^PD;i>5iB{xytZ~#`o@*t$MWiU04YJuxG(lckaY|bf@saK8i@hOQ9V*vN0RLn<$Xcf78)oy>NHB#^5 zV6$H%$8z>p$Lx^gyb@@*v~SP_O0aR(=o_Xfu>r-v(=ee0#}{|=kJhWsNjwsx2zW#s zf0yMQFM4oq1;Fenafwe9jtm!zpC7D1xn&r`ml)&1jBRA?>ID~ROAMc_g24~jHuzhA;|1S?!9L& z=Jz|QBF8n=mTA34l^FcQA);pdT!(tysggYCy$OfWTdGruBLFeBd1kILpM(Lbk` z7zTf)0FIHuLrsobj8H8-3}(-@5o{Yg+hIR=hssuvo+snMsI5Gb2NLpXXW-RU!RoyW ziv30Ico1pv$Iq3DGI-fV2c&f6)DyuDPb7(aOo!u;k|mHG6Wr|u@hVwYl~Yd!zj+@) zBRU&B#VtFP36CU}i`WNICV|4A?m)T1KOs9k8m>BwP2Sks{ zZoI7Z4L}QYJb3Kb0`XeowaeJa4|$tEGZln`enC ze>lMG&@Oz2f-J^k5WiLnJdWVg-n5>ao7O+%QFudkz>?pmDIPY`Qsr$0(uYXyI7LyG zbNub3Cog*=Zz4=8m0ms`j*AMIt{oG-_SNfR+1*}UKtxCjq-!;fdONJ5y+!Q|?$ZU^ zk~Xka)ON^s4&wt}Oe}(NSenDKGv;USFSp@k3)tIC@8g$kA)(?DbgqTBZ2x{5Slhm@ z4YHwxS&yEu@iMz$*&(HU^>93AZ*@dYv%|TBDEIlbv-EyWpu8CQ_bszM8nRIrJ(mij zvhpGdi)*y;EHOMHi@#HIB|>3jyO|pW5=(@90+WP0p+*tnQk$D3ph`;v_G0YjP%eL= z-l1GlfZdv%R@$uD$UJ{S5kndyGOfVvSX$(0jp3uH3NTrSxXg7B! zh_y+c6s3n0B$QmU5V&beIfwOGL0-!8?z2i*@De_Un>uVw+m1h0Jg7Ex%G=$MbzuI5 zD^?YX^g9G-^rA0N@z4-dU3A??#mFxRW1Zr8n(#GM|IQ$J)>YdOA7U8hLO&uKA|?-M zorVvN;$Tyr=wssImEL^$-10lZV%NH%K}}9s0aZ|Z`-`Vsq4~FU(s44M1^5?YM5%S> zWn)8)86IL}UNRHx!!IFHQh8OoBq9X;h+bJPLpcTkt5J_kR;H>>SBE9bKX~D;I{LHw ziHiF5y)+is3sNbMr^xb7JBt_KfIzy?~(pU ziuP^C)nnPs3o&2k;&ns`eToTPW_5~({PB;vH}rEMuQ;7|%(=_4ABFU7s8V#3R85dn zjgVATw6wJNl@k?s8b9(JT>i54H%W|)E;!Ux3e@O|o%7A`@!nfAfrOp72pyg1~^VmR=Z+~J|_gIP>6y1xm;{Ln58L79<;f69I=*uEM z;aOnhvT0hWZ6ZG(iLf1fXe1Qw2#jCiv*>H zDPp78E!%Q=$QP1dimYEvUdMCa_o~Yk#)Zkmalq&O(*u4ae=ggtKok)fY&Ae?vMLQ8=`BNvLF6H;ck(Wn&$f z7bIUjV?nwI1C`7~7u2^IUN4EI zY^+~tCW(F>u5ek|^~E^0B|XEDdXx7 z)xL<8TJrz0NkjojU#CJ(e^c`h$a0YA{#q;5ZwOQGvf5Rw*Q#><%D@P6_9H0jjFq|Q zulTzHno9$P52vY*kGq=bk1R$*3281n8ZxQhGQ;w-^gNv)qC8Tw&lu{(F2oQ)wxjl5 z@CjB3ix~82plquNJ0M7P%f~39TpwK@vx_K0t#d%UA80&4vO&2iaqZ_|K)iOl(8D73 zS1}(MMN!97p?!ZMUA)|x_yrk%4r`GCbd-I;Zt#q47Bv0uq4I+I*Td_H%Dc&Z4l!Rv z`waxEVk^eer-sZ9B&wKa01b}bZ}ig5pSK!;b2~|=>DlO8CL&wROHTAY#@uszSVGDt zph(4>cM)F8`owtqZkcE|A`aD~Nl<78LL<72501Qx)Pd?i&Hk|IM6LQ9YEb)&e$$XK z7e;35O4~fT)3qg-bcRI*z9B>aWFGkIB5{#k@4V3UyeeYzGBs=MOdUfSX|MX1e4C=~ zw*+l!po72vb(y^&r^k(Pxx`ncK3;EAKcLJLBQxUsVftJreT@Z1M4_NuV}3(PH=xiV zHoFiuX3*migJg9{fvR#u5CNSC6=$yb228K|+6DE7%6d3j@LaHK*F+O~|56YDZVq9` z_YoBlf@ypUR_;6eKpa3FpLE!J>}^dnqBH*r>fLTnCZoxp0-GHRY~vErHgsmF6r2y^ zN&NYV`GWuY@yZeJ%7Anz{e!XqR65_Z)f>?ZnQ-%B;464@7j)`Jlypk27}D zRS23kHo1$N>eGBrp*KaYX#7KKa#emc>Jhz|Y&4xOg5tIN3ldVifuVZ~?5XwNOzX<8 zPuyI-ylHKpL^lAnqIWG>HznZx;*Mhf^kgMya{>43o0TB?Z(A7f?>o(N*nZROI#gI~ ziJkdj0(v{`b(sqdcCg8FvN4Ke_QLZ`s~$fI3kskOnKL6e7MLMcP-kcICZkM7j^`6= zRW8;imn)P8>E*~jVV6t{vVFufYuHBo0slCB8ZBA#Uh}Q^i@T4G99!M(fEf!6`%nCj zbJ!UKvhO3i$Hw7%Yeu{fA8(1ZOf6eH8rIIp1(!y$^G`*z=aV z;NVCg34VscOIL=aEJBZ;yl=2ZY=-g`ngMgoIcU#8LpDm91_0D(|myd$Z$h z6buZ4BDj$9Sd0q20YiQwI3ZW+ul?eflN2b2+#9XZ+L_y3BB6{Xb>H+IaIo6{w=E;p zH4m_CeKk8HHLW3ep^83x*hv1qW?Yw&TdX;#Y#H=0&K1f<6OeS$z2=y0Gz^qJP&+%?a#~laX8$XEzm>f2eRTnTh=*8T%Bm zjeZ{1X9wFGTAP?E9d{gk`w~W^rpiXov`Z76wrw=ZpYANl%+{BXqlgA{L5xIpP;WIs_Vi5ooQc9S1cCQ3b9a8Bm5Lt@ z>{Cm=EXoB*UBcnEpM`~c#boocT&-zM=jo_6f(G$`VOnuwQp3SX**t&*QDTwwc??3W z=s8CL%qudP+~hGGE5@RLJNg`zNBu+flQaSxRkc3xj}hN6#^=upq82mbPikUa>g|X| z-=`@&nKu$}_txM0m7sznL#W0kC(Zl>2TE96f+>FoFEQJY*H>M^JRJ!~-^6>X-@4NlK$I!*IP|xsz|IqoL;UW{{ep*;O!SJCkXw4*? zYq#RJeIFn2%GNlwMHg(_I3QdCDNSUm-v+D@9!96x=-cO(nPYJ!C(!aN%3wrsy;-Sa z6yUwyFi#i)1&}lT2t)Q!@FPT*NO~q87#UOZw)c_o4nOenyc86HEdghr2qr_4NKI2l z7@X_8@B5tCTn|(1`K-pqDe zquhxY%-r_LNxx7mZ2cJZWiIjrt~B$650E-#{E?YStiz?bh&ZbDu0&{sQ420GGag{f z*Y}F9H|X{Hm;42LEg7$FX3S+BUuAFNh8f3_Ps)kF#^-^rzyfmhv`ZB_${F;o$GIp} z_oXe(GSWztDy1J^LiYf2Yfu*~^SF zY|#sKjUXT{NnrsJ_skB)@cHz^UK8nEhTFZ8-28?j;q1%hK-_I`o~>(HP+C;60>CxhC=;>9L$VO7Ir-iKe5#m-Wk z=D4oYd*+iwV4Dw_VHG9O1;5r*K;Ra|U8)C8)Q<%TYe$5LLT5g?LNrkDdAD8_6H1tj zFE$w6#XDsG=&`N@Pke3JS5WmQ2ZoMCr3*m{hEX=qxmI07uDmfHt3to@FNLZ2*;By) zJ5CZi3U+mrfx@2m_w^R$zj7J2HMFU*?3D?Qr6I3(WoHdE_H4oFkkC=)UfsxO5xTRF zXpM-br@;eiy#%jS%Eh2~HhYkX96QDlrC-Xaj}|Q22$5Q(>z@YU`evF4yA>y`0D0-h zp!`4X$x<|Pz@Xn}oKcT@KL2S+x7pcYFu9KfW}aG`WiIv6QIubaGcczjn7{Ky!Za{h z{FPs8qJ)RvRxGK!!3Ev1DsMfO=SV4kgHZk&>?|Y`%|CJ#q6g@p-i$>Dz~_1d2HdpY zUoH1|ylT+y4<|b|JCtrGE8O=G189DfYkeiyulYJlL=n62utbJckyBu#kp2$g+{af`iesPrp+(HW2K_XhkKH$ zG^TFO7!f*I4z;B1;fOOTentNONLdmG3FSp-iiENwh4k$Kngw0u+(P|iSTE})xS+F_V*mgtbh{~rAv}YJwHFk>@=q?G>Rw-G<~kreNgJm6>K#(gs8&S>)L1^rhAw%cLD~_$V%Bo+j zYcM_ji94wf+N@IK3>su=ku9X&*M!vj%KHj3S~)Pi;G+H*vCRqh93rWjC#fo` zUMh=#CIesPn@6Bfo(Z9UH_wz2u>q)?2x!n9U~_yFLaFk1QiYTy6zKt1#pDfTJ!w60 z)8hkNlb>S{`@|4k_1Rs@WobiRTj4|ElP5N7uOoe+Qau?TX;;6u5sl~LP=It8KKiv8 zT<&!0-=cAi$AVPsE}dHIuE&RHoWrAM`fG^GzFkq}BZ)FlsuyVGV%TWqqi!cFV}eQnx|Iu-pfNa(72X9h*)s~LM`;nS_62b&bR$IsS*`a))=19r5P zX#497qZ{#-JHR$aLGkC?Y+Tne#>=$q7c7@QYEXsg=Jx>Bhh*>jgZ>aGz$luHlO!?V z9C;u0D*ergNP3b_Y7R^Lw@50PJ2*Ed>}+<4h;6}iG6K(3(u zrrl?sdXSQ%>rp!9=J7dy{aqmPs_Tg{%?|k2yGdG>39ni5h>vVArn9Ub2a_42X%l3# z{WfYd3wkqu_ZAezvoMNCJ6U#VqK?h%lkku)-{cs5%(B9Ao7OOeljMSY;Wz=XPdFC5 zb5!#UTxLSRJY8a$oFAk8B63{&fgSI0M00sC+eJ39%t-PtyOPC zszy<`xsjFeNz&;_P^Tgon5muYH<2=6uz;wfHmm4^C4wJC1f9Lq%?qhN4o^-BHFJge zJdNOw#GvRid3sMq5u~Z*3IlV&i($IG!WRGLAN#)c#itSByhF?sx(|4ySTRu&?$t(N zVjvQdBvj+3!}DQ|JE`GJ>YuvhA}gzK=A?l$t*bmCr@RhYaQu%86XeQd)7eNQ7ov^t zNacBBM9$WLgZ2JIt10!M`uo}6c<@mW(^x6)h`mqVm3=OhBJFEsQm@}1H(#7Za!b3n zU6{6h^=9ww*MG5ES53b#e*elJ2%5+C?fLJ8y5MVfLr{_aUD3CR^)D2h@dU1wB1oFo zpYeJyKMAMv9OzrewDkC05#;6#?#sq7&wkbw&fX$5(uOiX0~C1*GUY$Xa$GgRMt%@C zApo3Dl25Q^hh)bR+#Zc=^A3w|MK;C=x05gAH03?jWv9**9Ngnfxn?V8%Q22M!9n`nA z?Ce$>*EF~Jlpp+2C#iv-#Iw@~)y42nX=xt1f>eTAUxG+CP3LEaI4lIa(&`~^Tny5p zp7u%(+t6g9e)3iPs<*OaK3I+}RZhK)fl0F;s}RpgMSh*n}& zhbkyznf-iGHFQ14k3U?=wOe8@c4E_fxRxlv$;V1Eqq^V&B#W0szLHk-B{qjc}Ry+kg{Lc-uG@#)`>i?ZEO)A5Wy_U|?k;)*7RT?@Y`^I8|yd2IE zQ{8ly1PEEMFb&_steGpE9EW`jIAlL?-7>J)is97K4h^!ZXZsmC^Xf&m?GUH*iO%;0-{QI}&(jn^O zH2^7l_N+BiGwW%Yp!G1$gnMy|%BLB1IfdMRndsA^g6iG!T7p>h_WpL8{~dJtmH&~t zS8Rge?;tCMNK~{}u+R4u&1aa))*AXzGL{4)%7l3xawl(gy%ZQI1>fTVF{J9}LEhl1 z$Pg6Ydz8CT5>9~$ae&B;gpz?j;G>`{kG?B9E}-Z`^E{9lXzOZ8bwg4V+&M!qALQ&Bzng3 ziet=Rua799*n_cjo~bYm(QbNU^B;lmfk1P<+>lCbsrhzTdwP6r17Uro+Mo+5NEM%k zevaKh%X)zL@<%r72WU+9GgF}21V}L`h0v95<`z-R_?ZiyKXGPlO8XXo;Oj}Mc#ZXs z$(xM807X=Eka=|aZ`6cF$EHok+XDaZP#GHyxxwhLKWUNB3q!{2 z;}1(L9+p1QN)=08Z$5uEKhkVOj1mTr4U^-qP#Lq%G*46(BBOy0l9T(Hrjx_^q2uNI zC6A%vQC4lXfyy;{J974!2*OH4#JF*c07qX|E({pVOpeeLS%W4p>6l3Fx=C|Z0uv`d zIf>WEjmC+Zk0ypoj7N~A@+o5!h0e9{Yudq9;Wp8sSFs;vQYvIpp6%S}CciMts9N~M zbzZWSwl$BpWXH5DK@$2yPv#B-;If8?JS^`5gg(s4s^ntPeoBUWl8-=0^-jorMRzO$=8@Yfc5$LvI+pnSqC(s@^^|JXOa zoZK&@oa;CJx&tUMw9SvOeFIO^@e_H;A5u`+eDIT$w3C9v{#QnrBo=_{<_qmW<0_wb@CKj>%WAslNrXfFLKdx1SVdUES&OzNvk>gjYI_VCQW^Q_IO zKCk&ZG_+BXpj_rZ7V~^kBueiYKBeeIOtRrjA4uMNeKYvL8+A6jR0B|d&y@(rJ^Mcx zogy%1HpaGA(5;1HpVhqRJmOk$O({v${y||hGhUxwY+jTtI2F>LmsLAbS{o~Gfv5!= z2e7=855?#&nct-~hn^@|=fSNAl(rsS>CT^SP88eVBnoy`&i*2OukqVBXDzs$(9aSL z&0rF#Qm*i|JD$17X-yZQf$yEO{R~%4s6+kAO;!Es>7>2vJLo&!iR19UPC^IGzTu-v z?T6oE$o<2u$F7y+n8iGlVZ2cB6-_*pB!XvnRY&c0Gw&%_UEV+KG2_>qdc1t4M%K@J z%hr5t-Q;5_lNqWaH^U{KsNH+Jd_H%P<#~qvp*c@ieD<%G)_&jfp}&*V;M-*)Emp)- z3yZzO?#S4?Uf83d|7;b66wq2Rk5Tx@kfUZ0TUK=(ZKnd>_=7^L2YepQ^x)arvTfRA zOmgyks;jesXj#jpW`iWCwIV1e!-y=aK82r6a3(p$50R$my{7oeitbpFS8n zv}qfVWYo}nIDL;U+N#AO&s5WA=J&jYerJy|fw4XNh_7{*4>3pJr4`yjuAQQJu{cVj z%qo|R{z~5Zz$^Rt4Cyxy_k08Q(3?<;U|X=$5(|WmzhbQEkSWFz&7&Rr`0L#kZ6dTf z$P+t?QH%eOgEmS=g-R>>*EdgMa9mk+aXTWMq2``ktOn=|pG1i_FibySz#e+KVBS&O+w)FtdFkJ zLRg3XYy^H`qSi6k>DaP+X~^wh&LL~^S(q!sQnXsIUiM&#x#rcjen=__IEJ0lelZW?oMWi=VZljMSSF%9$xA029*i`pkIQE^Ju#tt+Hl6YU5AX zw-R8CW7P))e3yJufA<#0|FWEAWT=GuFnnwH@{JUFWLVletb=!p=MwqR%zRPo*A|lI z>BIn*6I0xJs3}WZ+;_Jw=Ck(>zql^xyos*ST9sK?mu~*{wPIOM(80zr6c5A)rF(DB znW`MIIljPa-Y?J$DcmIeCFSKesdjfGsI9wjkgLknj*dqTHb!YtzBaknMZAn ziP}{)kNt(hO}wIb?UruKIL`Glru>zr*J3rDc&$LhKHBLMfe|6Lbn_C|3`$+xC3WQWu$UmiM zl-OJx)eB07Wcf1k`IqTaIh&oC;4J;_?&PH^ckm z`XLTmh7&tFisL;S%~!(tZD?Gq(K&Bh$^3{qEOaXeC<%E#{S+ih${*?ewERCfJ@nVI zDS0VIwBsbNvpj-@O(E0yPZ|FitcZk=_FM)}!ai-EfOpTzyQ9(+y{1A@#g`|;%9E^r z5$#Uq{tNqTx5L}Hx$AApNHAkmN;_9KQ=GP1&gd4xU8h$Q2?-NpDZ>e#=SycvK z-QU>kF|XIif;9lOhHM6zUAGQoW|Wci%HzVTm^o785ovtn+vb1^;O;R4lLj{e= zGnhAmhKNfX@SB0^jW`1@RD9&SZjuvAnz^K|)^o;f??1AdYkKdI2h7~onh8S!HlxRY;$3j%!qg8UlbGiY7}VUGZ{l%Ajxo$sp~Lw{4#ST)WmpM z>T_hOKxVk_5#1cxQoGYfvYBb7u0?8jyT_dR5G~Q~FRx!mQodh~>5%1R4}M`xttzQ{ z?=^Yq8GDqN{Htbbrhlo*VjFGrb5rz{mMM@#Yi!;7HahVarl~%jKSeO3D4!>VLWta8 zD2Yi_i(d$8Gqx_KBch9m%>BkiOP8BDRZDIIx=1tRurEZeH%%_OF zr|T{jqOUGY(`p5%ba-oiKP<&>9EUBFYm6&hlx9V6Y`o4?L~U?zY~9zVxIX^Hpl^JZ zFw5>wTz|5K|7cf%7n*SXYhA7qp2@!-z}I4x zUH3hJ#Ifl|B>1fQq@}=t>s=@^HDs8?q?UX{U(8Q{>QdI^=fQ@-60Tvy)qH}}9hU1j ziG$MJ86`kWOjynht-BJAgT;U5{u^uAHqL3%5Ju~oD;y#j;-LfNHKTMxvw0zt=}oI; zBWzsBg(n&wV?tL=@aaV;3)zR)rym~IKRaPHk@7obWSy*{BG%G3a~F&pqFemQCyLja z8ohWaE8Sa!IR$7T$oWRzZJC$~`3{&7oC<`}aqil9r+gn1HHy$CZ^h?kfIq8Xbc?MS zpK>pR+6gg;>)jye$6-$ui{pD9>9fW&Y#$QfrpYOij|U2JJnB08WZdm5uKCz2))D?o z!-Yq2KzwZa1sz34{Vg_%5@Y)R>FX_^s%oORZyG@w>AsY7NOxSML%I<`N~Aj^RJvPQ zDGBLrq#Nn(kWL9f_~z>K=33wDVlCm`6LaQF?Ad$&f5Tfs?2Sb`Mo|6WA#aJfe9Erx zymWERAM1MKDBGx;l zuxQ7f)}NO5`NV4~!6Q1?Y9y^b`c}S|dm6Duvn{!E)7C5w3vsZ16Cw;%HW%0CLzB?x z$+Yq9+oZ<&y!c4(!|T`;u5?@EcKaTq{rE!-$%t(ur z*HNZDM&CrVuCPjaex17tG28!c4-GXYn03VpMtpPg;sU##??goT6FionfrpSI!>a}o zr>Xcs$r0j_FUgWp7%8s>lv6d7u+f@5oYBTl9mzLOQi-vgpgkrJv%k4>t~tAMZs8u~ z-^#Cb#fRZxd&8Yr2E$8tUbDn6Oci*a#^v%&i{8x3r%rdVU|7<&SC3jQ@4uZxs2xo5 zX;KK0tV{C*|C#=FGh^F{k^Tvu*LhE7I5!4~{#mv1u^YwP45`{0cWNhm2mE$<#tcmf zUD?W>roPEqJm+-VQAVUmEZ}2|n3)UhB2`%%$}8`~DP8%1`*`PNI{A0Gb3$l4jg)=X zd#wgU*5cu~Y9FLfJd&FFLBD4#s)EU<&>#nqVP*aARvpSzQLT=8G4~b9ceQJa{e%9# zKQitLe`%z*K9L`cW+V&vnr8Y&u(gLTBH%}WSo5bx!6hg?WTQy!OH}X}!K5qxFWWaA zFuEMjBc`*kh=Kku z)LsimY<7fngmjB%0`EvcL#7)=D;9P-rCdv&i$eag`O;)Hr(;4_s4@OS4xyn~$)*68 zJ8j6SLgd^P1)f>GDEqtZme|Pr@O*P|=~DW;{2O%yMZPgI6<~aAq&g?<|H`tL8)RzTN78b`!f!Zd^5<=;_5JWtr-*U>0Ht88L2L%=X)N*&yJGsC-fiu|DNowdst1IFQM3 z_29<+GU636b05)qf0MrZ%piHC^qu6t(P!`Xk^o+gIbt3axwc&3JbVd3?34t*1KY{6 z^*#Ve$ca@Pn`9*Kf4$s>g(9VJo2~~Cl8H|J4eD991gJ*>`x_3 z|6yDiu=Roc9`2A19~}ucghV29|%X+9@_S%&AjlTCwBZon6)cftzlr}*&yzvdxnT* z(5n(cLBKy>^k1uSGW(kH!e@&!`7ev1k(^D^x104n=%}hL|NcpV}w!Ty(k0_WaonQ2XDXHaALxe3!YjrVsEHSu~{PHnjTD2?*^-b^Mem9s4!J`V@97S z44tB-1f1y#g<&q=Va5Y9#2ac}7UI*qcuF1~;-ditXbxWmgZCkKQc&Z5zcLQX2w7SP z3v?4Wl5k>;AZKen-rc<%5Mx-TpmAUC{Vwcpz%)n-5n+Ug54V*j)tlhQu7viTL%6vp zRBO77!N+*9$!A$N=fyN`xO&NcZH%ow(9JUNmp})|;?QlUXy0l_Srx*aQ(OKTwQDfz zY^D6maW%OorKX+Zyo}`q$~9oth&oW7=-ga2lCG;F*gxw@M@vVxC5S!$8Z`?qhT)>* zX~gD;*gQq-Je^?aP(**}hgwT!rh+)q_3nX1%*|{l^X}G{@C4=H=Jz`(f-Wq7uI&#u z2qUPJ>1V&)Rxh1;c@>9>{V$oyuHe4iWEljAZbenJpI=k0j@yJ&eTC2L&O~cmH$zP_ zdk8+`TP>Kn_yli^n3`Xs<#Ql`GpqVeO^>%*jBR<)e??p4fKd}0-3n2tp!sN^PAi_! zOo>C{{x*)g_ZbVW9&$RSh)iNvR-4kL2q)a#cVZ2;iU+cjIte=k1Kuvom;S-fO(3yn zTG7U=D1%1{XgW{UZli(JXv!XF7(HH$e`sKSCH@01VQ4IWRk<&Makmn z6X0y(y?hcaG)!?nDJYq(DpdHax@@ygh$)v&BTWkW=hd*0E9Uvcl5gPJXSkQNr z_{@G2#*OG6Q5K?`#95op>VtLCoKmYSXZY_L4oD zWn~cM8rSW6QP3=-A(qISh~3e)(y&=oKyg9TWq}l*`*9gQwqYhhw=6E>2Ek8cg zB*(>FWzX&;KJR7kK4h?uA@fP7 zQ|~jyk}J3t2~vQB6h|bjun%mDdstRG5{SmTm6=UeH9^W8^)q^bd5UNWH@anJk9tqD zVtiVmDi|E@=^F#>?v3=N+WB}lQyS`*HihFYHd5uEaWgvPE8$DfZH!2cAf+*11aV_? z1+4cXNQPB1jD3{xAKpdoFI`CsxJLfBzWo0hGyYLw{(n!?l7qI=L$a9M$rXkfmDOxs z;Kv$3%?m>4WHt>Zl6XV;WolqnD> z;3w!Gc=?aQt)!dIQAnU$!QT^K7lqiezVg%y>2q3zUp1*<%wItDsYGgG-}Z_A)yk}e z1NarpA$`M;vuC&s($FAit}K*2WB*+wmpEU~VzB7O&&iD`1V5y^7A}TY5jU8ZJaha{ zD({}|oH*ioYC(HoET_AUgx6)EeX=r}GPr2reOEGF!mn@UAZNDk7LE~@Nw8ntBAmFI z4e^;Nfo(1Y3hLdrLuSK;R&rovmIlxeLQA+EvxkCuq-%~2IE@ zAXoOJBo|b-z}d1o+$b3%FC%^MS4CVQh@EB{EJLY9C_dl0WxP3eGE;qaPmv9OP2*N6 z5C}Zf#s2n6)^2d2B-3o$e}pp6wQN+aGWDjAs&XCc{Mad;G_=N6&1?2{wTqWhce9C1MwxnEHkyLg`)B?uBxX(;^CLa2 zsNDU;&JA2WehFoRmh%-M0_X2>S=;e&%R&Sqxn8XHwm;y4?%IHJyFB84)+@AqMly+j zLOeuu^=DS}tCR@5xj**S#vkbe#rMzVU{3P?XewEpJs7Gc6)9K6v*)7oxB9C$%*a-O z>QieN`4>shTwL9-CB#p~stAs&8|}YtvKWKR{cuj;5kbB)z*ps85?dIE^%GZ-dAls( zONd|653Ug6fpl!-JDSVvP~27019Pr;6H^m(Uy^Ig6AH7Tna6@E@KK(vHx2pg_Hd+X}`|)BL|C_5V63f#5+^VPQW2aTG0@f~i*@ZdBxaD26NF)5}|i z;j31$c7^lKh*K~~^0a{u>5E8waehd**pE9{O77;X*(GyEcm};7F<4C;GLSCJ=_QwU@ zJSqkm(ZE@&yF$wBuB*UilarGJ3aSu4fq;u^V`F3W<_!?g7kov^Yy0du{)22AIIr=e z3al|b(RwEqZ)V>Kc@>PNeoN4oo3kZM9@IMGB&5h1l@-chw;?-}OdOk}oak61W`oC$ zHAjJ$61%19y1aaMD8r;>5v;0GLiqi(d1wATbL_5Yen171>Y;F#VvE_uhYdKe@@|yf zcpaq)QsCb+e8rnvURLeffuyKP?b>1?ed))jFemf}tA|=4lTWl-3R*ytt$ljZiB0w6 ze&6IF6+Abahe$X16%gYS0*Uqu`RFpVA}F{dxG->TH_EP;*cIiK_9!!_(SGXZCRfIb zDwu05%v=1-oL4lIwtT^Gc`#RH6(uZrESV?=>c~MvnKeQ9}P|(K0k6=k7#~M0_*MmpGI@lp@*btz|U+pk8_x87`c98JM3}*Fg9czYpWJjQZW5r#tqLe2}6kZP+Ho3+>uG zEA#uapYAlk#KXgLQtI9ko|&21{~Ge$?|#Rx((OY=8x*R>eB$0oSCV6d$!eA*tjeirZ>0u-8WeD zIUl~;(`j43T(IBy98CEN=*xXBa;UcE-;`|u)pXjT8xYWezVgh)&8^b&+yX`i0R)BP zAG5Q4;30vod$IF&`KpyR$`h!w&tY#Kh6L%Px{ui}Ys&;O+VS~${gWF4X>T?9>Q>$^ z!D6!d8#cys{^ZPo-_tk{srZ#)HJChO%7!kfXJ6>9O|fjSL$0KL#|zyN=vB*Kb)%D# zk^=gNSO>V-vkfn*-($fr5(p+HrlIFnqR7MDxqrs${e|j70zjZ_oO)a@?uhl`tBQ2I zDQn?tIP4X>4x4G9PdR0McPrJ5~O-6taoqvOCd+U`L3Y$}hDhp#D&?n#CmF@{n$geU`4SsTpP}1$y+89nhR%l%LJJ^U>Z) z))Aq7)(9@=`QwV8^sVZCq6NQ)7*tRc~4g{Lhydl{pM!EiJ#uC81nB17U2=i<)8yfaT~{&=t48XvXZ+GYCP zd&c$C`CYmD^^&bz_w+AMoQ6Hm33)U;JPfz!d$m+uH+uYF3C5|!sM}Csgj|Y1{%KTc z4tRw36xTM%F)k>f3SlpM|CGCEG~{S05Lo-Zv66hg4r{1em4oas07-FaW0)%Fb}~yl zp0^7tBjSH1#7|}YE@WXakPc)bhvSy35y4VSGQN}8XrB)NqC*8>ATUPBM zNmxIR@LNM1y`n)iPym#YQdDSnkpg(U0DT+Xx7ny?Y1;p%>$JCyz zu-4i1Jh5EFPtf&{{^UBjVt#q(GvY9Yj5;ft*C1I5I65HEOAS(~1qEb3W89l-j6qd= z0OtfGLaL1MRw*bYZY&n=96f}J{q;#A9l5GTJY~mzWp?fEFJwyb`Zs9iL5@OD76qVw z46rzasnZtW|N4@l=4o6OR|!)=0od?u;&0C}PH@eJsvvl9nYghgN(+}CaH0xMUq3&Y z)>T{Zxt8*K{HF!gws)^%EU_Z2=*>X_iVUFII|sD%r1_XZfpxx3K~2~^Iy-cKx(RwByO&jJSx9v{tjCQ-;uof-BPhxD)4lAr6N>4AEF z@{#THl!(rnd1xt5pDmm#5$2o;A!a zFPj9(0jaFelelb)B0K%bi5AE@>OIxTOixNhPKk4T$ib7?jQ8)~cOSGuKIF*nDP8vs zdbby!>MyFGMn)jS3~fHzEC>QZpN83bN7`ty-&PHsDMX+o>@W8wKvw^!!Tk{lsNrn? zu6Iug71di&v!RTh6gKdKUz1t%kt@iM#|yg zA>d&}Mw0QX>*(}$6PYN<%Jy!00pim7S}+yLDj4uB02m371VBx}pbv0zba;k-e}3`s z@xjgz+)0=AYxBi!27`%v%f1n?0vraI3Qhp7!`Er$;X3-EZ%W5x7>pQl)HpA}Jj!aG zF&RiX$=f3WC&Kyd@Qud1qt4a~{w4dA8_H=kdh)8QfHNMX4&L+EO$3bJEM(RRjE9l+ ze#{s3%NHt~40Pf!y(}MB6#uA+5ogB9bkN56WJ(M*bHSWV&BH$@BUyLil2pzDQ5(Y7WgH;{X!*wrgF!lk)%O(ue z(9f8~`r9WeI7uiqCu*6~{f`Az8HGsugIih-HpSIK@;T}7(;u1^@U)r=!+N&Yed1z9 zVofc;4|L_1)z~P+-uM%;e3n!PGl-T?qud<;hiu)c;rgenxPl1f&S@`|_$!0t!S3Z2 zZxWDUaeU>x?75>{OC{tI0_+OF+q3uRBm>kRV49qzt=t8{Xkp#^znlOX$@aXSZVF42yW?SFE(q)O(=y*+A*R%G`SEp59SY%>S5 zeS8@O7AlK&Np@4yZOGf>#(nm`#H$Gv;mE0Co%2Ff-!S*q^9&eF1C-bafY-Q%ga&@i zn2cJA-|7KWpp}&sKqo34nf)~#deX#dd?2yZ@%^hVBYwZfC@0CmdVz7%8J9X}CRX}R zS=O)B`X|B9`Tn5fw(pHt8G1T}MV0poauM_L{hJ5Jp)Z)R!4G zC3zn;1P27b!+i0Ek5 zqm9*$_s*G=t(rWN!;&CS=RXt5$Qv=SM!j8>b#s#+N><|EdA2GS)IWTVshy{Z#*y~pVX{Y~yk_@h{QUiWW zqfk?wr5+)mO5}Ifn|DY+ z$pPZ$sBc}xM1tL%Wv-28k=w^sHWUH7kH1%Pqx+}-&tGh7L;R{S1t5$WJyxFhz>Bq(SHmWu%@`VC=egcOEzvCD-fj3IP=B2tP$@YlQ!|c z=9DCgm@eswMW1k)lhv)-ZJyC-K7M~WHaW|AdgSKT-=ALl1=+9Cq`*(dCSW(=cq>Pv zl_Q(?eYC@Pr8cW0PUrj6DaSMUwy&JIhc7x(e%3KZ|L;!?tY>3o-d2uoCar8hDYoN3p( z*5`W8uS?M?)YzCIV56#c?UAVNFRHn~PEbTRa2nwbnyn%05_F~im}dWERq5knGOsJ)`{<*iBGtZGVjo}qaQriaKBla|3L{&KPr2|!) zK=mT>y}iAk$HxWVzF`JSKRnN}?;&146#$T|>gp%_@4*^XaDcAoR2Qr3ki!;H)t_`7 zivOD^U43h8auUGph`cWknB1R+2Mun0$;?- z1!(^k23D{!s*VkfAUT$b&~7g93!rEJSr8ncrxjS_T`fmnXvc@?wJg7QQ30PrlqCKn z`D=XKE{)YKQ8-n&3qHU1q&I4}2Owe(LWMN_-$Y^zyUHJkcq$UN<0u@i5 zTO~-o`!uO8gfgKCtldKy$!LmT80o)Xc4?3p&Iz!a;EF!QHEhkk1e6IILU1{1P+6QL z>0mc2xL|%ciFK%}g%Nn~%;3GpL`1=_mNMDdiZ4^Yd*u}hb}Ri`Iy$pUp8m{K2>U6# z2f}2k0IsjD&i^RF`VEqnfjRa7e%A(PE?%r>2)Mrr2JAi*R3PDmQWGb6LXwY=A+5NX zBodfAcp!`Cz3nUV!Cj~!Z};1jGKRGQXs9(W8ZgOTNCKomZCD>A?i>q*D$Df78YYbd z?#`TUTZ3WlP{%H#$K4|=&t51)$fjaJ#KiSU)nC0Q&?p9Aw!1TUMWds5)dhOl1#F0attom`bgU!!-6=Ok%i1{VR}gn8i9wn|aZ&MHO0;Kzb}WST zDMKnXpl(=LScLVrXWWI4uHNWz*0!s(J^+2>l>t^#M?SV~41j%``XvU5cLFm> zDmas%`5V`^irEmM*-+B<_BMjMi5PIg$aja8Lg33=TByMLX-TBZc=o}ET?*h>7H)TF z)bkRGRQ}p{-3c>;0htI_;>mAW^hF3IIR(NCJCkQbNqg5Gzfg=&>Y=2=JrM%wG|56r z;3R!;5=FcIKZ=n4OM`&htif6WHryJHUe736$(pGf;$bhx(2m%O_;C-2v_VE$E*Ep2 zCT|+AB#9Y!J6rpE3Dgrrvmtie20f@9aL?yK#*r6MeAJ$ghgzC&TInt6yE~z5`@$Nc!2xrcetwB zMNQS&YINwNlvL_m`Q&%68#}AB&T(b>ziy}aBq1S{PhA3n!H>&!1dhmD7qNLGLyA$t zd8~DCnqA$Rz|IH~_>+!<3=rZvs^FYHZdtfBm6ITxWK@=9hK#`geych_Hb&r)q^Psy zel@cuO!=80hPZR>;^ilv-I9Q=lqImgFZvjg>!MTWBPWs9QE<~HXrV7y<2wfO`?{Y~ zbJ_fl^V>8xulG&@L~+&w4JvH*z~V$s;Jrh~Hz5hom-EM7`1I1I(Il&^UQk#DBIq`v z^)~qRbo8l)WL*-mn&?}3N&}}OIgcbkaaqP6Oo7O+BkA51;Wk z4&bOjvy-pGe#~G+`+c%C56RUEAIx1n#2tZQuA$HmK;OB6h3Vj8+)VMDAm3zGD_}cAf zY4;Wmcc`2-B0xysgIqdD^AqeBjE*94i0xaKYK)G?VN{lwqV8-e1^DsgjL9r6VQl|B zz*=4DIT}uP1_^EO>Yxl7&KCGP+CCVfo2j4vNg(vD85TRS1qFOrlR5IhXQZfa=pi5D zhJ3S<|E5A;Rs6_T=nX=B71t|}hX~L-iCE7#zE3N&E#By2)a{5wKz|xLQGiuf+BRo& z`~+Eh#vxrPY(&J$5&a7J&r_yB%hXe42Ramgizn~?4mfa-s6BCl5=3&8fddc%76!4_ zj#Wc8+v|#|8Rz#RyflV$NEPNqNIyX3g#u4hZ%6Jc-OSXugFz#F2~N72RN&900P@|6 zN)jaoy-FndOgMH$Iapx0u+u7#S(`zHA+1wrxgk5>d1k$uJokT|5Ua%eTOjTBI-PK1ANn}_UO@R& z^)C4#m5R`LnN3RE)1)&bLVadC#;4I`JUQ2J>BenI!l3ll7RoRUqB~qNBANd_)V#BL ziXC-&yUh-vHFc#H^BSkTo45+C-2aUtp)ZL!-V1IlY~#9O^IMngVtK0u+PnlWqyKxA zUF}+m(E8R^6~PrUo@%m<oz9*WS$Vw89w?t(KhQSHD>9(LcU_HDj2%l z{>SRN)Mue&pV+K1U=$HGz?d3rS7ntCB!>TM_$7S^9Aq%dW#d=|0%*8bPF))nafcef z18qcureI-}Cf2kZu^%EQ>UIo+zkDT#-M179`9Fr>@66G*%qZ+sNoM&PVq%%8)No9H zCp(6}`6C1x#vD6+gD$cLJ9sYvJ#dUT9_1Z|p~Kx;k_o#t<5zX38w6b}8iEN}<^Vpr zyqd6o&0j6T?0fQBWdlnx2eWrPD)%@xjb1b(=u#xTnvBR2Y&w^}ZDHF}PerR=j!IJHg?8 zmS@roOPkNV1*sL5U8Ouk|ZZy zk0R|U-1>fjPcIw<`T?!7nv5Ldms|>CpQCXj647BIi^*#_Qtzx!=7BFZHh8j@->WOY z_K66Kk$>K=n;AE~R%~z7Dfo?$Jv!NTWHpI(r&@oqu9Rtx3j~WuLH4y29|`T@$+~Oe z$y)BX<<23DJH-aB#Z18%&nuv)`_xUyUHK81{|Li} z7hn&o53Z=jPC%cQwt}=(h=vdoVnqOqH6YE;hP@)=0jY`!&o_sATcKZJ8=AmA*AxlE zMWw8>UyfQkT|i_0*Bw%`(Gyg*A9s6gAJ18+D&qu8Fy~7Hu8{(Z{F5(b6z;cIdcu?P zFAMNTes_8@$c?r~@S%M@6z zARty8b=zSetIg_!MWn4P3KHEfJy%aqVDsIR)NBi)BP4sAc;$F}MwMWHFs+Xw0&OG+ zH7bB)%FT_!M!(Vmo<35F{j>&NLl((p zYXM!)5|i`MA{%5hw2vQZ-lH!Gar&QbO606|pMA{OFBo_0)|V zBV-d}^9;=f=8?2}r$Bjg=L(m50p9o|L`M!Jx8yH7=wV$vsaGKydmZ7)ymC)_@Ii~W z*3O28gpBbMFY%zFKqdpZYL)B6ouQwzx5(;6%nv77Ab=>1`7c`qw&&1e+qBvdzSW*P ztJZV>a<2TD|B_AayzF;Qymbidecyx(s$j;|_(b%;Y;ilU^F7ao2cG_onZFmapO(by zNr?zqL=btT+YWNjz;-s<-T}lX_e@Ro1lPu54Tov9CCohDm8@^%GraD^X2&u@(N6)53UN>F@1IES+_aNSo9S z5K`pSLcd0BYACQ73*{Wyo}mq_Wjy@w#=i?SP72QIxb1sWeun90I`5g%0oH`fpFmfQ z2a^ejVI~O+PolAj@IiB$2kXFN#M!9{zngB)w7-V|T;W{q0cFBq9jEBm019#QnUMFy zVm>3RX8B1(K?i_|p|1qIMGO?Q8NBsG%#yB0-5!CPto+*Sk-e`?7vWKauGsrd!D>Ps zEGK37aB1QZqw6drqX3=vkz9I7XJn!>0#1y;Qoglh6xh=w5v207#)Uqz9+Zu=H0_DM z;Ap~NA?T6ElI06!6b(v5pJ+R%gdxNfw9vux%k|QllZ*L0)|6RKKWAV6ygC&jh%mX? zL&8rT9e>d@6x&<(()F4F$;VE8 zM1TAR8*y=hemYpSC<5!!(!qkZbEVzpN;}%=Gr)pBe}Bf5`VJA8f=nqz0R-j z`DJdlR!MHalsBLZ?zzeGsmV8>l6+Z<43ZWh$)J}6NLiYK)`3Dvli)edF0u`If8|yR z%KZf9dSSOL{v?-q{EPADCmw|PSSv1>5{SDcKW@QSGVOno>Bh&J!S+H~EAI%I z`S=X~=k-pysA5&A&{=;NCRbY~%-fxh=QU{EJpaAC*D)WgHTi%b?~F!T5V2qS-eQ zbCwbE@K*>r$_6{ks|!7sbZU%{Z*Y)Gau3v#NW31=2W}?r-6tf2!83Z9NXV-fANv@3 zY0{VsFu)v(^}mc*gU|bSutdQlT~yrIG+0L00XaE2D|LzIB$(uUlEylMX9X^t$q}0d z2*U});a*Tu$!8JPM2Cx|aMRh>16Zr$L>hJlrQx541 z9P3~zY^GaR6K}omjx*D-Lz~o}uV8hnNCP&N)8*Rw<&<5}?#hm|Ob;nDH6is$3-P(C z&oO#LarM3q%TTORV%osI?Wn$ub?q7f)7&}892yMl3&Lb6yh*H=-$KB4d?1Cm{O!EZ zfxhQd1V%L{u%t$zNrqTrBu%_+!*U;KJrp@X-s5`Y z__uQVhV&sF#PPVXAglub#ra$o$svpBCilD`F%A7C2M*%0xF5}E)fjq6v=oC3yJfi{ zWfP&XRrkTd$b0`$w1zP-E@gL?tgdZ(wZ<6sAXvQVNsSV-2f#4&4Ni&WtCAreb*V0=%)UlvL7yBAO7%7)`TiTm91 zC%~wa0+x?5aH&7niL)KSi6;fCNdFo6D=VK4Gm`Suxt6vxBs+3Qie|#nZ4*iPz%?~& zO9hbKHvyYnd%s_HRz!hwI5Yb|-nzkwZms1AJ>82y?Jx|&j)SK(p_QV}M}fUx0I>lK z3p_ww6B8N~T2N(f2><;QBP$Y@y*`TNC4Q8$pjbUy-0tahb0QiJRgb=h?tF2Fi z&4FM={leKRX2)x{5j`Nk91H9n2bvQEoJECPST~MdPrJH!%ErX~ z>LS_*RVL|l-gu|9dgobA#pP9#j`wAROSIqBRnyg_d+#jB>RDHYR~ZWoIqA@mF=9#S zAwGXqso@jemlwa;f;;p0NkX`$pL+zwA!Q?q`&pda@?lH_kpG*~ewEprkUwjE#75J$ z@L8lRkN$b%&zuyYBOu%Z2>Wt`8Z{b4v1FQ^$m#A!d%2fZA%cD^yB`PuP{s`QCA-Wr zQ|#6A5S0A_iJIqk=01*2@Oa_h`i0@jzMUTO(j!R4X=re!l z0PY@GoJH%<(l;FAu7E`Q%}(|V;awSOs9PM<6H~i^e@9gPy+!HQ4`0~qWSzqDJ%~Q} z;EppcVs#Unu?w)MJ+^WS(!(4l$7-M$U`jQ6n;T9!iJrif53d`bPc5EIO^XY{XUHq^ zZ)2&g?IV=Nwa-Y*lElGzn-+sJ61+1%c-0s|v4?I9`sr-x!N#07RS3LU%pV)*1nqsJ zvgc_fx$m`|cIlo}+69|Ma1m+#q4>D8Y2Lq6x!LlgkzDd4qH!v(jMXOpN&G_g`KzBR z=q#v4NC(L> z&NYEah=z+j!t-AQj`-H zq}XD}-T2wfA{yXE1W@TA#;YvtnC*|s6`@BOQ;6q$-n8YP&s>pDY6jY{PSCWLXd%Px z8<4Yb+`eOaC3C!1 zVD{E7*dl_9S7(^}yj%P6&pBIK$ct6kc2Gi#3jP;UhwX0u%4vLTcQ-`SV}EeMG?bLo zE`FcL&ilJGv_vI2L^E)^)Lord8Df9>gtdN}Xg<%!ozz!PaoNbo1_y_=BoCF;OohEO zP#?H2@9AVY97^WjLc3HO1bkJs{Cti+8%y+?K7u|a;C>=P6DX!?#4h$TG+6#MMXI#e z^Ihsx3dTgHF?{SMihox@wjnB)gieRa!PO(z+jsC?!PjYPpBtgkX}Ky5ys2q}$0*#( z;EilfdgaS0ELoxZW%Vs&^<2U&f6-XB7Q-cW=m}21#<;v2yRxHO92=t1zo&QaLC7n@ zau7a&+mgW&SDX3+aY_qW)Sf%oX*XTPIiYF~W;rZFS$^EC0)3}*(^xo7wch;8o}QO? zK=W0Q^aTC8tBWeSe8xd{>#wwbmbhv`*fxCK^nSHEryUig`NDl>w@BJ7LJ(VPX-`he zPo?3}u>I}t=3|2EKFF=;_w{oP*Vl5lvsq0EkxCO=im{W%wM3<+#g{*h-5efYxcB}% zjyhdR!ddF{*4@u?6Q%>rmioZhr#-3zZ8|vh`t60+W$jW|P^rZ>H?kXs%H-kC+=6eR zKXMK+=zFgOCs{VM* zKiNEAf?MP3{*}4~^FNhjr)o-zp#9owAHB@k!zx@WhcGsY)F452>~o_J-dGcbifhmjs2xdS8m8K^Kp6yaEf8zQ0{aML@ zk+NmRe}`tQ4;IK5;0@g@=*{6d38xlOT)CZHoDl1w#a?|tGiL-W$Apb#m4BVA9D!H3 z!*+y*Y%iOZ)BzlP=)i{`&y&%Lq0wc{PH#11?Iw`6^mlaojy`$dXin)DZ1M!e;#lFUwyVLJW z#o~>*V)0x(>60n6UpG19Rs1_TWl5CHiRHZo*O2y)&>Af_Du-y5Zkqbn-d{ZVu(1UZ zaRJjP*xFRkx(~s1aiKq;lNO1)x8DOIfTw#lgHt1nujH-o$qbL0CDcA*$&liLpxF46 z)Qr?Xn(!;dh=q`ASpGW3L?)k*56P=IbRH!IRVki&G1sL{0d(4 zfcc#ZZd`BVDpH20$YCACsu!*9@+Q*NopVab;_p{-l*XUSt`Sq@71A5IL~=!bqal(n zu|u)s9#)mIH*4M}ovxOZvp~P#xn06KOh0t?`k_@f&UHq;JH(1L_ML|;>kpPlHTLhi z`B!LWLviCVv|qm&G0Kg+COIoHd#&goFY>UfKk3xjoEk8}CoX#3ycNcj}>;dE%+2%tpGTqkcl~Z-|bTS3M(4-|p<%VyN1DXpzk_ zD|%8(0Mg^=^Bh%9BeCPIr!pewDn0N7^F@a5B9ARynz!T2&+_bK-IrC2>zm9R5KL?m zKB+&}^W$LgrR}oJvJfE1+nrVU-ftT6YmQ0WOlj#&3WBT83^qxs=DcE+z&b7Np6P?L z-f#CmF$4z5y#c>BKQ^_AHim^iL(5lC#@L#6us)Taw#-_E=6|F)9T#p6u4!vacGwZp zy>VjE?n!B^E_pn;)*<_8))RYq@=-<=>h49&dZ+2og8Unv4;$BlVm=`RcBuMh4?AsP z{1!(YR41ID;T8IMocHLJ{lm|1(JUBATt%K(2Fs7~oX2(gsSbussSVBj93a-``oUsGCDAz~mRDZM6 z;P2CHpW-CxJy)b)t&^^(yH&sEU)`v@=qzE5TuX*vpEC!3u~ordXCinckHbVLw>6&2 zkOkrLj9|=3p78YGH;TTGVR}%oFyQEDJHtcYKt0KS{!{@Dw_Af*&Gx89Y`;ouKWBDP zafZO&a;ZI$S3D`S=zbRy+T5=3*Km03kz7QXQa7yJF3UvpmBpWDIj_^`vkZ-eY=y?x|0E6U=E*dS^YyKB+Rtj+ zEm{X9%qrXeaUfiApoovRtG_T?B z80JDZ9krHR6ub$L_n23yw0`vr?2p2I-P_rDdw$&h@O5t~aRxPD-=EiV-koblKl|Nn zq$=bS(&r1#mQOV3*C%QaRsTOfiXAJh+1bUEYd2n}pOn7T`AJ&4?C~egxW!#a+Dsly zN}Q>E*&FYhs~k9547{?p!v-z#4N|S)oE{(<^6Xt=YPOS<@B}g9lG%y$f zC*$)Jhgvq18r_2lGJ3@XfA%Ev*cqq25Mx&LDEOCN)-N`oW?fE;u|>!1ZP+VoYi|%R z3MkrsVuH>25M)Q2AS-V1`X~)kp!yc3+c738Jw zw52cR7-K@<@U=Saq&_dQ0-?2mz=6;b-_(JDo#qmjZSdKnqOf|E7q&;;@$zaB-_}3y zcZF@@BPr@REkBBrAIq(>oj;?wf3-@mbGDIOa(_g;=pibM!tvTqAk_X0jvtk(K&L=p z-9UBdQ*Iu;6#JY~eadsBC=~wj4Ger23;EvzgL2JzPNoAM>u^XJdilO@Iesd7>t`J+ z8f5*_TRA(%$Ii43eC2#-PoOjWv5{6{rFo{My@I)}&|)QZw{iW@<4m3|K6`RB5A2guGUEZG1A48mt(dM&IU z!Hn}AhS~8kBV0uK9ZKfBpVd=eaw#)6_WUjZ?J`=^1v5^5wVx4oFJ5swt!JoPIn85h zzuKQ}eZ=xBEV>0vl3~^DzFux!v8b^j)tAV*)LHH1d)^k2lWV^dw_j@!#{&jpzsW>8 zRasM_6sYlOoh;uS?{v`}!KYfTuuU=G2Nb*8z@=n1YZV@50P}+g6B-;`XXOa5YGxfdtE|`G=~g)<0qY_~?d%9(Lg#H6X4f`R zNMuz0{a)bT!Nu0bNr&XHJLkO>jD{V^YWm;tuXVDa=HVgaPhzDZXV!Haj^QB}p?%U2 zZaN5f{3r$QG?-91v-ECZ&#lA`2Pn%92x7CejQM?xp>*t!tB}6<#FeySRt{rjq?O%~ zpLs^0ol&!fsvhrJj-DtB z%H_3CpNh{M5t2URctEq&8>_XVWbREh@K4sdys_E3_R&t#3P9jP4A%TrUbG!>jL6?a z9?zobs-$(L*{b?NZ1zHJq_>VG&6muhSQxia7U~ti5AQzw#_nfxh}>hK!N-Kt8ZNd8 zL>X25i2X<>6K5HTJO}>#OHmt2roos`qsStLkBOFZk=KW_PFw!0LvQvvgegRfQEyRo z>o->aR$OQT_vkvMXr!JZdI7pQ-0rC5yLhO(7lG|T zf5NNPf+06ST=4BacOZRWxLto0LJff&JMtINe`e@i-_p*0?%(QJ=FNoNOBNS$%|uji zT(}p_GK>!yH`+Z$iE1LqQZ&Yd@$vF}(63z~XGW|XwCy29M1E7tMy)5Ngib)9CQ)x^+2<=}P+QItk&{_H7 o-_EATP$Kca^*Q?gM-Tt)x7-;+lY&2U1PA`f%cww0rCtX9fA+Cp{r~^~ literal 0 HcmV?d00001 diff --git a/_images/poisson_3_0.png b/_images/poisson_3_0.png new file mode 100644 index 0000000000000000000000000000000000000000..470ea50b5d63b93a0af7b1360696e18181d6a987 GIT binary patch literal 3679 zcmds4do)yQAKoHVN0L({*BZ&aj6p7ui0wp*ju0vtG4A)<45w2`avPW9KB+OrC8{xQ zLx?E39AhME8l4V9(d&5VBo=86x=KPvlH}!^(Djxud-q5H*tYYRAws2N z)}f$Dl?8OK_AoyZbSWswGP{1YC7{7?ffY1=Qo`If$j_Q@Ds4!D~{on8F zg+}=67pUA$gA(WfZH~Fdl`hdkufS5{`#&;V;+GrUCdbwGD1hB<)O}((y1Pc@!n^JE zy7lSSEXd7U%5AMARO)hCWJ{;4yDpSQ5YAV8cP{^UWLV?ZZsSkMzA_3wTk0$5*d0SL z%2e^&n`?V=8VdI2SllC?@|lyS#z#(h4YAGlqb)CFBi+oTNj(#hBobwoy$R@DbqSCL zfEF`rDWF>>5RlAt7PShyrT|=}RLMwBO-{~tbaZ6IT>(t;-eUMm(r*MC4$TM{4T=`y zaG|}&MY}34BN;-lIebABxVoPP>`<%_PnniP0AhAT5!DuS!`dOzw_w9JrjMhAa&*DA z4CSx7x(9XgqAtdOFKYdw$xOsiYSqIEc4NP_QcbJ*vM<>A6B~sr$mI}<&~&=6wwokE z92>L-Wydz>PZ-0%%JD}B2-~nWb?_?;XZ|enPJOVT`1M-W$cfFHD2FJGvh%?+t5Rhh zoMOH5qLTqNJ)t2X^=et3<3|GFsKcaX`?c@zp)LGotv5_fc!@FTOMZSno+;-1{mgj8 z)0w^fY}6;1z4h7UlTkb=@O$J-y?MsaBAmT3ti&tIGt>lodU_PYMKu=!M`~TwUeYyA z87>l~GhD{bs<0nx1vpWCrHaI@_VwAV<&BUR-3vy@SCa7hm1$u+_&25o*$h)`LD7sl z+KIY)LR$O|#`icqftw~mPrhq=pqgUQ;E$7^zTSJgW|r1p1w)SydNl6gCP(vxR~F9! z+XI7$o*hkJ5=N(|FAR)LtJbtG_@1UZ%OXVp*Df<`c_GZPjn6PnI5d5z(EZ^2lSOQS+Y6{zs3iO;=w}CHUB5kPOv7Sy=IbcQuLs%r zoTegl552)C&Pa6(x!a+*P>vr6CE0ZBv&C zoYvDD)LXJi@Qg8RleMAcCkhS3vpZIK#>$}l5ro637Y9xw zlM#-Nj+GYT209NTX-IBf(NM|c`oMt08R>K_=*8N443ik2T#fFssi`SL6O%%&{Idt> zEMn~pI~RLEJs>XbRc^w0*Q&pOM@;qb`PFdy+gTh<9BPuJgF}sew)b(66|r}?WJj}~ z;EpSew7dQ&VL3HjBR##NdiGO$azT{GnW~~)>gaB|&lS!EBK?(fb3*XSs>pEKrc^tG z8+ttiE)Ee<-;qD+$p0DtP&AfAfIHu^SS+pG>1}9!%+x(v9hrRA!op%;CgQA6szN;b z$kR)wd2bUN5)u>rkO5gPh}enyp3Sa0yQ3Fq)iuIQ7Zt#O$z6KAFgN$i6|eq$kTyDc zdZklL5|MIpzyser4mbXzM*ae|=!A_gOh05>%+l2Gqm2zr2ef`#eJUizWxM6%5GE$X zAAB(GSkA|O6_$XN1I19B63u~*a1a>Pkiv6Zlv>|%wGxGAz5%vGd^SK^FIS-ne z2-Vfqjr~AoBUIds%*+GaIEYK<3Zi@`mY?4DODvppK^lc3SG2XY5mFD^J?dMM-oHX4 zEVkpDnvA5jZzouYr-(s3;Wa-HEbQIH1HoLZp}F~b+UsS)Z_S!BP}>UZ01_Mg=M?i66u0|B$tUrs*~;4Zc#ct`EjwvMl{Dl z5!Xrx^UWdjfj1N3!O8xYP$-mRvfH?sEM2e*Y@osA0oeVjDm$nV$lfhGy1C8S6IYwF zMW2fZ44>{bx~(z9ye3fFs~pDT zYtw(sc2>DoG(_QtO+{Yb^ZHP^Ax@T?y7-t=RR zz$@*aWDvjwxjz{nHS1txG7>2_R=%J0D)S@$OX&kkMP)T>-T7l+dVqAeJ5c#- zZzB@Uh}2L#X<0nETxD^tZ}DxWr9V=?a1vTQx_1fV)YrsiX(PEPNAK9qBJ|{-w;{|@$ubjc(UZ>%oxmwlOW%~o<% z-ys3TV%|0K)qPs1`%O%!UGIaH(pQAed`@(%Fa|!M3$iy5vBEl-lP1!+hG89o#9<9# z^Q-r5$FLjSgBUOBH`P)FhhszR;H+%UZM+)1B#h@jR=Rw!=0T|NI(`IP+XLsv#|k4z zFN2X)W`x)IlWTZUPaR%5e42TF~9pC zE;aKjIO{M8nWVZi`=E8ky`SC4U-rmb5k3stC~iS=UtsI)J@Prn!Oj`Bu#D|$6)lN? zo=7}PJrbL&vE-iO5w;*|2jcLRjj2GOEioJxgyyj9*pJJ(I#&0gxwl-f1!Kyah!Pcj zr^CG0r1Fe4czCg!Z3efBx~oZa8&+2?znwZ3(G-*0{I_j`Z8_e*u# zZ?CjUbrk>rN}z+SI{+wz!`Cy4%i;6!g68{hx0JjG^i+h~3B{0~;ro?Q4nAZ6`1ZRm zhXP+f{R}^}h_&;M^&sJ6DTiW?0K`MFQ3O&fA?&c`u_H0$Fj6E6X@WE{&^#I&8)a!^ z^zZjbQcS2(md2%2_y^`e+aEpS^F#vDQGaPt%T%X%jkdau+UBRXg&f;ktF>-xr>0`P zw?52Vc8@>HU`ip7%5s z`l_y5WD;bvu=@eUjCPQ!}XW2cyOU@fijq`BP(IDh`eMfkZg%S8LA=g1A z95t{_XpU$y5W4mbG<)i^ybZ9gj{anq74BUqAvtuQa0`?G@CzLED{9Zs)Xmt@D7Aw8{PqwFU_7_6`u5mtu_g7B0NLw*Ah%x&)ZogNp{yEu zsbVZ5%mGSBNU#PW-ub1s>#`o(o@-1@KHVn8Q{ z7VFAA6dWp=>s>%%yZ-?6QWc(|*HP^dqQH$SJIS3SOAms9*Cdr(o50zp0E6F`-?gJN z?&xHPDvj||a3PL9{`%ZQBEz6LbLG9>vGER8PbCO(I2?UzrJA~%u#a8fXsgLeAjsvB}uNBc>i~d>Fzt<-N)kACW2aUp<-31>PUqwqvnTf`p>0d|0Nn)_T z&B^zQXE6>}e|p`eb!RxVB+@>?4n~u`YeC;rof3W1o?pYL)POT*&U~(~S1RW0 zBjVBc*}IH%-K0)IaziUNrJXRydmI`X%BHXXv3=0nJ5J^<5F65$Dk&+AO-z(aHvMqM zG#mR?>;!Rmq723?~;HlJ;qR_pR zqy||YgACD;ZKh1ig(jk7c!c<6R(CJ6xxQl0dWC^b&P_9RQ`?R(nvb@yPuCwQ4PvfI z66|XF_*2TPxI6=FMe8NJYg`lHu+$`Tgx_N1WC>4zqcYn}kxph&3_P#i73cYiauBSt zva$zVz<_HG7ptcmN4jqSob>w8Vf)V<|GNR?>v$GvU_;W0SHMkHm zXa8WK{7%rPkABD7sdW$iHSX=zY%fmVhDdU8AL0!K4$X#&7x&LSey|{k8|@D7LxBGN z{wnn4@JNm0!!n5QEnWdAKTXm5W_Q&82Ts07S7b|q)%E^*ggPXt=4$*PcMx87-Vl3O za-p(ArH6kVPB51wL8D{(GJxjDE?kTNU9*p;DXGzZI2vS8_28!iNR~0K8^(yv`g4xy z%K@4%#ZDv+#`khE&SW|2s$6{3-jUfl#rA;7M^IbQ(7-4v=^THX;c)~*S^7IF3}mlD zfos|Q=x~%30Y8j4!&+J8_8jUn{R0kdRI#F}s%m$CzmISyKsXhL>key_U14Z}V?yB+ z0<5u=l)~DmG9vbO8JV5ve(}QFr&9^=V}(!7zn>&}xcia&>KJwVVBS#4Q3P6b*eWvz z{TjK$@9ID1ED%z;Sb*Rc7TGDY0$1pwbNhc`kQ!v>Z5VG=m_ce^qCs~Y4tK6Z;p9gL za$gIH_(&5t+2bWbhXe86YyVLH|JjOL>0q!OS z5fL6XDQ-)cmR((4FKZ*Ref|87;ID1nhNkq*jQ622n2H6Nw1m1f8w+^m=bDpBSu9qm zwjqeW1ve!*VmDL4vpD>s!@phjo_VjxFe`4sMA|nSb->VmOJ^`8jRF(1&ks zjHz448+Q%oTTzfb)NG_bpWTTq%MM5Y%_9fhumw1K`xJ!msw?(69g}E6OHqMcJxI}@ zGH>0|(*+Al=n0NRMMXJzdELY5Hb^kxw{zc{Sbe`E9Ply{t+H^sdi82OuCvp6`cg;U zt|A(3GaMt~L_eEzVy-42QG{;#^yZ@S3FN_U;Py&Nz)1YZr`u+R5+?;|6f&7jPhOmJ z);26&T}#6d@i^YZHvBPztEl9TC$>Mj-kGnQ)kyUF((lt-_0B>G2zk@t*4NX+Dk|EW zj-6+!s;QxO?P7IwbnrTA#l^~~vJ43hqC=*JLVNsdrTghK3gJjXIJq&pScXH)6RCh@T~b zff>KxnNG#fg-3_Sw9*RlIqE_cn6K3_Ndeo^^Ei{_(a}+~g+;^7X$5AaxYC{FDcso` z`~I1hx@D{bbTGe5VBTF03v&#Fy)!Hmz9Uk0vVX;!MdF8H_GPWO!l0>rTNW8uI?yC0 ztEsA*EUa4s6oTx+08t0AIrPk*($HP`Im{Qvlu5aNOyaR{0>NM=8nE8Nb56H*Z<6l0 zPBY^_F-_)&fq@OfvLvW7GOAJr^YEY3(}%tsD0RJq)mgtHT_`^v_3+L(CWBIA##WV)R47CXeA-=ksltZWY)<*Q{V_MHw zoLDut@59Czz^kOL?%3XfH~ykh=@Abg-r-;AUPtq@>%}np(V$KH=X=2IB+Q675tn$w z;O255B*jVNKg!GlGraQWyv3lo@r@|i{^HPYY93xpWhUTz^@Ut9^~QrD;K+TyC)MhA z^o19{6jgEvF3eW&CZvcNVXx78D@|diE=FG%0c}KJhQ;zwd|-D)dhWmuAqotnNZb76 zEx^H+Vr9aL7Q{k5mjaH-2ZL9{j`w|}g8ViJOO{50a-@)wE!_-B?8S>^P(Y?>gRaMt z?bU^kk4dY4`ELg&$$(iHOk^ZIUaA56?=Gn z$|rTOnS_^b7Zx(ALUw6j^f4`rM`@Z+wbDs0u=bU>d^t`S%{|2!Q5Crj`I(b|lf>8G0 z^9nCF`2Q*GXa@Mj<*ISYkQe;(<-HOD{$B5L@}esQ@ok2GP%XW;mcif=w-Y9A2F^q` zPpca?&^0SJ7e{9|$E(&mJZx^bUUhcD%PGhy;C9%#xw#PJ<$phybG~6KA1Qf16oN!Q zojU%Pp;zp9ue0OR@7}djquk58Gx>urSLp6NXs@;IfzZB;eBGbb55!_#MyE{`x9uUF zS9x^$T6^+?SF<&SONoY0ay|T86}|o*IvdvOn2f1;x!z3Qb3EH_{J|G!Z@Pj;>^4V% zj*jaG1(xT(Y&Vs({-Sv!zRyo)XzAoh-2-Zty%Byh3k#oBg*5^~LYOf_m>oNJW}OK$ zJsuP*oEO5YENj1M(+$(sojbn@OHcjfU~jMC=jXRs()9RO{?zjFpN70v!;a-q9}+XO zva~HNEm=?MXz^t&lf$X#wYhIoP8|PmRWB^$;m1VbP1vaKiAv-};=DSsE$+(0kdUv% zG5C=t&LfWvLqbAma#^D(4UX#q(3}hV0_3GlrFXiH^!RTt5m23s!$w{78Q1VF8W>;q zvv4t9@25fw(o|JZt z*L&8f1yx~#Zs8r?wfwj%*oAt_dF=A#qzSF_kk9vtnuf!DV{v~4+DPCq7es2AAIdy$ z+MeWEjMXdWsn~^A;bxRTPKsoI-M6P9NLE&szTVJe@O_J(!INp-hbITkTd_g%cLj}t z1V>Q592HNnK_x=+LS+{9Ht<^q)zm)6?RfN!U~jFdsmY#tP*HIJ44QvRAG#}e6PlIQ z-ziUaA2pif+J~FDG+a2}dm{kF4w*Yh-SA)@%29D-w?m<=eLu&pXExPn^xj%lo8qdjnO~ zgb*ngH#dKCn)~I6v-byR57xnBZd!k<0$a%+ud?k&$H&jJtmT+zQSZPI;nZ9I{d!eT z=yH)o6l)U3eX2QeS6g#413@2$dHVDzef{Q8Va0=f*g)i_2ouTj;xq6L zgu(EhA+nt{$v<<>ZwsHSy)_1dVMBfWOZECmuF&IQF&#^DqwJ-V($eY?i-4A|0Vw6= zeeCsXDHIraV8Sflaf*$6SC=Wvgo&-~Yfp}?k>1&E8_{T=-6y3@TdV8q>r2@Y(N+Bx zh50a7t;G*J&9WyO+J~~EOi!n>SJ91cHIS^)2#YZ>FnGc?h%R;b4lrc-Ae91|H)M>)E^q=m_-oN=^qkBPgCjEz})6c8yZOcow$>*)B#j`zX^ zHgR%#OFVtpD3G(Nd4(=j1@p6A$hi_a!PLi1D<{Qs zxH#~B-UW?9Jx;_)T_qhTo?2l7Gfp#VXluke*D_6BEL1pKSs>GCiLKjHcf?~X8YCgu z+igNU+<7H!qk7kWqg~g%zTx)0YAjMZsp=|c4;>&cLQ5!fG~*dIXoXzbWCWR^E)CQ_ zz3YfG#kG&-UW%qV3#;L)ezf#Ad)gsnBE$S8pwG5ppk?~o8x_0{Xw zWkL{>yODkyq&Yk9HGI&tQqwV*rU0q!)Bo?P_{&2VVjrcIFS|P$Mre8oFH}epY zQ19N$!nm12uc<^&4k#WV3?R5hFYQFK5-Hp7H3F*AW7R(tjiqOhD$9Ub`K>ZmsxMsd zW~KTZtBfn=tghZd6l0sDo*L3&rxQQf{$;|#e3S^v>>Qe9R zmP)YC*L`d2GL)QpSo<}#b@L?Gnk%CtBh#?5L_FE89KUplmF{2qhN#;wTI|}j3xTM= za$R^k@~-o3&j9nziVDco%&ZJ=2X?E?U%9lZs@d5?+&nzjCwi18|4lctun6C}-{|?v zm(r8!Nhi*pIV|4*%0X{WTOWh5BP}fr><=`4rm@b97d7lA78n?4l4pN`?yq~!Pw-=d z4xHq}1#_Oj*h~G?_D_$wA-v|kr^O?aTyV|P-;2Oi7~_2LNwor6KC3{+uP%MK4N$G8 zXAGg*r%#`LXEfh+NS6h$sOZ!$fPWFt<`t0a*0#1UsjuTtj~#@$z1;i{D42~2>XXQ3 zt8w}xS2gf6_2q1(&rD4q{>(P9ojP9 z=Uk?2SULiCHC&7^Y>*;E=V#!F`T5VcDx?<|6Ij*78)Fygy@edegSPH1cJCnDqM=Ln z_8WGei?4QX|J%5kLYh$?5v&vpcrn)9PxRobc!t_8XBS#}2DDS291j{-C{tCT7rbt0 zXE{(lzaX1jyfED55~`-Y%)fCX<2H{(g(muF(2vP-$$J8r${0n54XY8?Ge?x|@##EF z0qBSAV%ZyFn=R1~*G_szAE%dftK6=Ib@kf^=Tdxjz7tkBe$jGrBeb-9$z_3Q<=FLh zeN|P}HgWMPz#Xu4H>@77o>{O3cjtE`@XX%SQvGut8ZOV~=cf@2=KOjbf+{ zX!3TENC9WpxW!#IgZl3VBF&h?=lsQQT5&RG%voTETW^G6H}Bzt*>uxZVqT^(Kj?UPI~o<+q2Kl^RLK5 z4=JEPE#rbNVS~gqc}H{0C1gJ&QTMmF=N7rG+<^U`V^*R2GfO6x#{DO|^7tqbk~|N+ zO~1Z+=CrTl^04Ku&FNWLphwlWu$Ta}9xBXs`#}BF+REteTlHvE5Av3sHI=M3%CZ=b zJC>Z7SP2W|b*tfrQQdE^3MpOdGR%&64Y&>ZaKQWToIE?!e#L4KC&;1l0_a z91fr%hYpLq69Wa;B*(hu!ip%+;^O6fQF2YxnwHNmUcTf?H+nPO7@`F<=JKLZ!ilR_ zQ`rt6#f4S8Y~0*h*8+!y-{IOd5N)p8u>PMRoUuJq-R(=0k}wPwbM&f zoSa4JH+J6tC|~utHrvnnWy3Uv%W_V5xBw8C-MX>NKZ9w@g8P|dxEp7)0(gL&7k9Pw z^?^q=zP>$wJQC{8wFv{MM;h`!SVAU$1gU4&t~KwBjEn|>1-feZIb)W~{R_0Hi`RMz z{{xZi%d7s=Fspjqn3kp(NU|V;&jqQeb--@o2|l=c_!UcNC?MWG5jmIAHXiC4Y~tIt zF@bXDM*pH|w39`Q&f$WF8Wvcuo<$9|G+cAdVP%a~Ze(O+;DYjt*{-f+5FXN*{|?V_ zs4)(7x!{mny5=%eR)qSMV83w zXFrj3qa3^Q`8!;aJh{8NyYaFk4^O&%;yGAGcgZX!gyZeqPr{v7TG~Ev3^}Bv1ekmN zUUgK{fcM|fO;l+Ejk{sYpw4zZ6iavtPA|DW-<<$AM;2>*5!51I-pfFP{~pO~`BX?F&Styt^N zo(po#DP7GaNW;A&(qbEqjDj;Gl%n|s^od4#df;4UGdBV^0j=06u#Z*zmfnJdmHB*U z`3#fhl~#diOe!o~Ys>Wa3xWJCEiHlku}mq!7XZg>V98p0sW_YF7})#WHb&g^DCrs7qFQa*?CKpAdr|-CK)HGR2MVRlfqk zz)kS&)0&!rK-1h$`hqMCHno2yg$~He0noa*s%vW-!D7R(l&}};m>!@4px@@@fsPd9 z5nzIgYr|L&M(@KakG@T(Ki3%2LM_wIou-QZT2?Uy56 z6%$WF4)91~cI}8fEelTlx!3!(SVRJr;#5;#s}V=w_IAyAL=jRBDJuSoaIA7&SsUey zkPyf;n;RcRtTT~QJKLk-!TcfCIWfnE*s1_BAv#u-23P(C5*Jh-sziT%kKxZro5ya{T0SFdit$uLB@TF3?q1alw5L>Xwy!Jim;?&cyI zCWLNh>D&}cKKy7&OL^noeDK6QrGKvZee=UB_u`iX;maUkH~hLDxhMb5i@+DC^OS+s zM`7@%lPQ4{%HUeGuEtACcIQsK-LHF1?VOZ`F>k!tYXSEIN-3HcI{$?FnL}hTNE6zp z$!jojp2r}N(E7faaZ~FWv0ciwzsDPDAx0{ZOD&|#FwSF$7lFyQKi^-%Lv%@vg+lOl z+jC&;&;eC#KcN85sMRXkJA0zKB6*T@@vinglB@zb;aO)8saG)P^K|$w{HP*1A^Ktz z>FE)7++ng`XZY@h3^|V~8Dd+c-+x)r#KuHAO^cSX`-elD$bYAbrMg=jiRYp_BO1n% zBrwEzx+wXGJ+aSaB-)mU>Fk}tkR`O`@sx!awVwcBSaL^bh(^&%3SwPJ#>DPfKT#jnmAoJZ+S*C)gblycPmoR!T_xRjCu-*urcRb^du+;YY@f(x#NGy4B*=oNW~*^Wz60omfIC zt;>p-4)vg(cc)j;+DM(Zpo9~HPY&1zk{WoJJdi{vv%`4o*x>Hvc1|z#0GmCGiytp| z;F>(%LJKNTcPv4!h`&Mzzp4?@dwoO276sc}e@=!oMofUSQ1Qn+&!bM;b)So)HF)JY zym%|D@Zwx9+!cvq&hi{CU8z+yju>LTV1+tI;qD+8db(7Q%Z?zX9ONHND~QYypWtT7 zZxVT*fx!l)3RZ?;XABl5rDnF<%nCqB(CYpl%2QkcdAgV=-;EXJwRVLX6mDongV&j} zc>_a!yI3(Q7Ej)dDqUYur=hm3K_23V2xMpHbxh@g>;7AxUt5`EXG=9OWev;P0}$ya&bHq7c@S{#gJ* z6Zkdn)69h$%FkMN7?`U?v(+l0GayXIz9-GaDBm*;&=pAgi2W z#KGi0{sRBqhZh(xF#Kgu$1 zdRwWX0k5BN*g?Qo`86P>{P?5^#H@HFJy|S2C0Tc7{Su_#;^N|8!lI+=A;C(rd`y

O1POg=E0nBb`m5jzp6V$?s9uYJ|9ZXGFs*pe0;g5udPXKY={W4OJ zj(IUc`u@`=f}Ncm5FZQT?TpfeZMTqy5Y_1=Lhnt3TT`_zpFm?{V~xd>tm`8&0YN}O zKxkOl3qiqV$-Slv62e#l@K-i4Gnl(Nd`2*$e+;Yy24AU~9npjLSnf9>`i0A9Ov0o+ zf@aFUe{xW@C8p2kF&I^^NxgZ4wB_|PAn&|-eq~T9t8}}kt_ln%@VBtrcMG(S<5Y9GN>MEvkDN-_Ac-k z88oB(laXo#Fwa>>Aq6u_HBwB5b`Mgf4YGk%1?ko6_)H8z^>>3w$t}Ej;S$A`>KTgS zPY!B=aqa$saL_Fxb8oWBOsW?$3?YgH1eN2DVF#;c7zrjJ2u5ctiaO)3w?MIYTq z1EP7(;&9mJk~I%J+6v+?f2QteHKdeFl!UR$qtMP2qWBbGbSgPaKs3eiRhv%|lnPfA z5JC|yRoyaB&jkaoQqa9LimX#Y7trYuugIaenVl}GRD=EO#Z|2h3!O{vgBWWh=B_3S zy>Pq-rycZeP!X96u=V-x7%GKUOs20dOp@3$932z6`i^h_HC{JK^hVEpUcWI-_aV<; zn5;2Z=C_*dcmq#^I>$pc6%f9knzb_6g29g+W-I=fBOFN4jdd5K$Jln3&-WBi3J`0| zCkq&CODx_xxzWPmzMT(kXQ7Em#>wazY?&P?&8z| zw>gR0uiu_qO~hw(5Y6zT4gcWb$y}dDo`q_~l5+q4I`&=VGYSfdn$&laLn1$3+VZY5 z4cgq`Bc5j(maqZH^v4rd0(YwG(&FclToniv2ZDoCf2pK1y z3GdpUA4j(uaG)%UnY~88UBsmdes|`$Li7;J6YRR?ulPYz1%mGxDC!kwF}Q9?WfjT`D#n3ArpWIB@mNvy;_3Oq4-A@2jO{2qMdNR`m!E1u^Z zRJiUZ&$m!ejJ8a*sP9K7%1)yMsSkyY-89`W3t;Z1m-}<{^z`GUnmjg* zo5bM}5eRM%7c)EK*+?U)uT9Ule;FI!U$&}=OUuY07z})U2@J-6_UxUU9JZq90cE|z zN(VxD(=JO_cenXc%bP${oUTeNO zR?YhL=DI}xT4dv`m$qw2swh&bVwsipRBM@b{Y3r;5)id!%Qtqin?hYibK6(>E`2fw zk0SgYwHTtCO&Gtj@2yqT>#KhD_5HoJiLZ2J?d&*nB_oacV_xWZ?BvTObNWALRU5Ch zR6lG!5}B^hA9G#yMiap5h@ukQ1{4U)VzSiYj!n)R+8u&dZ8`JG=h~kO+8dLcEM2IO-f**(s##-3FqFui z>w2`b;4~>ed*C8+yG>UwGFhZ@udPB4wuEtcv(z^`Q~7b3gmG=@K;=B4!^^tX6{Ju- z`-iq=QbJ1@24C-&W}h+FRy8x2`6qk$kwvBzY{v$+V+bSjZc1-2_8sCsV(k8vvrE!# zWO?9yNIBupz`o}8$AxwO0Yj}-$8ugRqo_Fjsp!vGFJPCU5fQoWXInz&6TU2ke)Qe`?u zeSd#X%4>!X?gCW7#KdH=R?#^*1KgnTARsCc&q}@3{$NOUH{9v9$Uthnx{n2B( zOj~^~bDqkB69U%wsI4rKape)FGFA1pFh*>K0D3VSw}AyBha$!)+3C)efu^s-D%pgUNwQZ`~&05X1`xlsRFK# zow0sm8k8`P?CGOychgZf;mql*w4e@+%pL$2k)k9yL9~)<)boiH;ot7Rf7mLBZ4?&^~U% zfU{*EX=(H^p)>5Fq9QiEmSodR@5BUF^=DL6BW>PyQJHuGmu;%n^A=j5-!g;yxEW6l zXx`d$+i9mMj>cm!BGb&LO#Z1k`T0kW&xlg@DQ8A2G_+9m`)jZCmgp+IO@9#As{e3X zlV>^%5ipP>=?@>)$FtuHo%9gucplWsrSOK;FZ-lA?@fPFQtArC6OnHBc~H2$)YsA) zM5FN%)6=5{F3OkS;I;L2Hl0lGOEP9=W>RQW{oVETc!j=bKwzNz=>Q`$E9-iyZJXiA z+E2MOfgjVN^K%ogXq~@C6Hd;0evi5G@CmXS3dxTCJ#;4-%)81#2RWNT+0^kKm0oN6 zh6KkA`F?nD`dI7}dGK7Bqg#YgKBWgZJq1ZgRA?`3wZiv-6*wuO%UP(7;|L$6h*xsN ztETUN;wq0?uc#*r;g-HD*B8ctb z>8{*>4Hy|2xm_)~?kqGqru$B(I9L+J6pnuqbFOF@9*rq%&(@yw(i>`}+Hv87dFC%I zEw6IwoDC{&Q1!13b=>{niwu}=L)3rr7|OCJXo_XbpcN6xc)jEnnEk$W9o!8eL&JNOg6!4*{^ItLKlQk$;c%ZBccnE{^!>Et_2GcWNLGCPIAT^5D;qm} zc{$I{T8Ell8UDV#2%p;Z_PM2I(scH{kAdK0;zAa$YFY> zi(^(+JX-2-px;N*Ytx~s1)dr@;JghlRBTAjvesyuYG4F z^gc09QXRcDgPpaB5L2X#hp*~tbI~ye1RTEZQsn5$$SjM>SINyr5wqVmW486l9DHvb zhcT#mBy_SzV&=>SGKoA4k4C@Rdc-AMLM6UOkd4-&`cNeK7RzsK%GOm$ORY;@!ZG_Z zVjBDnNofr(*2Y+ptI zPcD9(0H?7*6n|_yS390V$Cifr)4fn%v&f?ou-_DZ>+nUJ3YTxObW+@|hYQsnnXXIj zApjoEnBr%dKwSAbflG3lseg5UvVq$G`mWHgebZnI4Ps(L-AkXDo` zQ54vHJ_znq3xJ!xBSMvg{0~VElk%^ppv9AufS|`(?jY$?QU^C`dRWlxSy_~m>=o{j zxbY;z!dWgyRK#I$-j(5h5NRa0Ba5dypI%jX1o#{u#9F&d^2N1jX zr!x8V%(5hCb<*_i!^57enIDZW#Z3EQNNn~J&NG%3!{x9 zcRP-D1m}%E-z3>0*-2b6p}K}|>~Of+Pi?@%?(OK=T4~DN*Uwo?n(Cbx_LtpmHz(8r zTQ~km6&lm|3U?TzcX8?94bS^l?KmxdGxfsAxK4dkRnJpOjf&s zTFyoV{{~HL8j&<+<${=uj8Va(mhOZ4<aC5mNQWlwk@$clZ7A9;iKu-H#Z35ZQh=3Pmi8CIXTzY z>(Py?!4v?B%Ld%9YU_yRFFFuMe9IGh#I)(LP7k<;1OL@c#8m7816YW^fWifOy~jeK z<0Wc!Pg=)T14$_k8x(w+tj)}xDp8M%i|g*j6NR<6i^9_!aqAad=1;Um@c^6y0Gsu_ zy`k%q^?`u_!@sfo=7!6DO&stE@-~OIj{1rjLf>o_n+PZ=hoRBJy)Eu%#lk0^XCtpC z0T|`w?Hzrz5beGm`MTlh@vdLc{)*RNSC{_@sKEr`p0e?|MF zR>_lYl>HvMn7Ab9spk#zJw=vAJRBG;CA>Gy5#SVfF+<2PimqnqxXpdMu z>uE$j0bE!B)2sFCekLTfp^YWhRk z6{Z$H;VDo0*>n9%yL5S-T!qdN+byA)k*Nn|84b#XQ}?o;bJeDJK;-`t^2GnEWzoiT zJnL+*ME7OERAxKw4rdqh1XdOc4t}_)|`~%V|RVy^ZE8yC7@TZpU_3NwaF~ApTR2dJDSy!B%e+7avzz(Q1mZlkr z>+0xG3c4N&6?u>Dh(q^gJwfbU$unIIlUz^NOZoM?*9B{`j3Q zfWv*fE;djG!woibFKFZ%3;U4a3loO+zc+Oz0lR(4%Ny@|zfME+{JCNM{9KJWVTpQ$ zNbW%^aBu!1&91I@)IQgSK3C1l=?>eb&INMcx+WDxM*#TA*&FlqYgdM_rvLz!!0(KW z=|Q&b5B#b8){xxNL%Tvy59oRH)P#=pA$JERfhHYTnYE^#yVn@8fto^1R2-?sh@(w>0n1F30Dqd+BFXGOxGfT4TwcC$FSIn{eSNmnSEGu}Z0S zeU!_=9|8h`&Uj|!{vpBY2y{lpjNy^j%W^6zL_dH2?2{sz`VH8Q`*wCnGOzhFAWec% z2_^zgBQ_?cqxEX(!_(t~`Bd?9iq|&A9exO==hTM+2k|jrZfa5+*tazg5wdMd*bVgg z&?q7%OFB|0f3I<<^oJF!mFV8&BXRl}QbIk$#_rx^6IClm7S88c@~Qmrqq%ChoJipe zPcP?5t*XoT;Z2e`9L(htzZfp zgTZ)aHK`&sT{(;J$&(eoL0`~r;-)=OgSL9!1P*y4lK0f&$Z8IxBUX}c zGl37pduWo;*&IyQ!Oz&hFZz7YUZF*Kztl#i+zV{L7`OLw?jh|~X8rn&y5;N2SF7l@ z&F*i~(XHWVLxnM0p&g{0s3uAB#{R8`xakQN*b0K!ze$x}fhYYo1w#DWKEvkNz^)B? zz7pa&e&V!5cA@>k0#eu#pfg?5Zg2l5QfOi$zc5-dBe)&y5^>$~Dz^x#W;0?k*5)l| z-h@9&g>VvS*>`;&evmhhR@EUAqcN~5_~eQ1ym9jT)VTMucwmPmzxWC`Xbr0D>BfJu zG>pnTrFqupKFiV7mfx$qR$46T-0>TR5At?l0+hjkU7U$cj-6D~1Pzfdx1RGjL_|Gh zn1F8vf5q*2wQc1u*b^5h4W8{JbVM$EmH$R$oPR3Vr?k-`GhTf9k`hPX={jtEV7VSI zH-E#yx*s*-nR{jvd13j2E5qxwNyPEcgn25VyPqy5eCg36TS)K8;RkW1*hk`7%eK%T z?at>0ndSqF*~?nVgPzF9$acpoZ`l!vxq3fpX~}wcKsq}+`Lxt+QD@NErDfqon*(`p z;#k_|#?pcMe!u1yrS_L0G<$H-%WpXR{IzYtv3L5x`(@)*CwQM~l&q}S0A_GwV?%@L zr{&GRM@KuK>8-J*&q>3MIW< zaBa~RiXjlOeNnRfg|(#leONIL;G}>kIl4=_i)}{ok?su8eDGZRCfd_Z9{;zVmm(C` ze|)GFy%NlwYdl_n7r1FNGOw`E)+@eT(7cuLpZXDhDy;<~|0Z&y&k1l>M%HMRjls9k z$1-_}k0jHmB~t2F;V;?7qw8f~1x?UZezL_ZR8g$~@p9u``^v?{XN(gw4A(8m! zVt-w#g?v?_mH?J>)*VdcRPz~3B405X#!rq&usZF3;^kRM7NV_-9A+DsyNl71a4HE% zjNQLC(lYi6!o%J6N|>@2>PlG54Z}9?^1RF*Z47WNt^6zUJs`5m>jQDOl_e8aRKT{L zTHgu!V;jHC_DrI0fpeaafUF`atExY-pX5dfPBZC#HTCiBue_u{`O+i@k3~#5=7f@0u&vm0kp^+#io=nIPDyaBV>35TI)0(}*D^`G;6e%$U*K(x~+xHe};bKc)UC z<2yv1J&MV?zp2Oc3x(mw8WYG6#C}t4S<)2uA&z}U&{*-#GcDAM9jq;3TphXqKoCvv z*H_>@9Z~h)qf)0L`nE$`N_J}3O)iR7}>ZKpw~ zLyUCPdlx~Jm>5^RPLd?ZN5&z?uyNDPw2f*9@i!tsc+wYN} zr`|?UY2i5HC77cH+lEP3TTG|2fWyFFzdJ@{?9XpdzT#>(>OqA^RRYS!!a84fh4*nY znu6jhxax6sL zZ+C?w-h&4peL{i`7xw*$_F z`qA5olkyKnbK75INd!v_-mDpqs zAz_?u8Hd*-c`w8a7Pfa`JV3RnSd5;&W?yW`+WQqW z$rW1)F=vkSH{SGU(5X@h9kXf$;0Zqt@tbmYx#C2b9AteaDOu;0e*4%aw@=aM=uYzf zEwRFfOlFbB?6j(!S{%%=FtI#A(ZQWIwpJ`rfBfBrYAejY2T2+v~zL z`8n>Gb07|cTbfS3 zv&l*RC$Bx0?kthl80z%`ET|JbGWxjAD>77XY+q!}&QJO0L0QoR`>2ORB#E=~9A%XG z22XUhd1A~|@Ct7-mt(3Stqv6WPc877G^m^~_KWO4g+Rp_gT-$hvQRlGgZ#b5V&l02 zxn2}{!>%>j;*cr(mM#F!0CcG!z@nia7d&9*Y~Vxh3Cp4iRLUIS=vn_m3RQVXYEZs+ z)z~r1bB`T4y_%bEB|VZ=0TrImEc7W=gnIrwULXE2#|M`koY(EO$+5pb|GM;2g^vX& zb&(qA2f@wD`CZd27S(!_sP$S@ zC$r9JHTAEG%EMzEnp9FnTI&i3(nxD8LeVJP3k#1$(<4BF04ndZdR@QsJc=mkWGL!E zr%Nku^7+cTr89lSOTa71s4sD}6V>^khPVQuxkzhO0YL-Di73_uyoM)-Ox)P`pj_6B z-Y{@+C*Sez3i{ed-+HJZX->|sRV>8U!!LF*M`O;3*9G}(sO0bER|AfU6guc{-r-=t z$~i__ttozpA1eqFrZ;$pU8zd> zazPxFzv%*fkbI;aY+X)`WoXOuWjvSfgt)@^B3ZZf1>r$*fxVD z7r(y1l|#w;cT9#W+o3z1pr_c@7_SNcG9{dFVi2CPk`271Y))shoj8mJ-QObwTk#q7 zhT;Z_di9vN(&Rm7lZ*OBqJrIlBmZMX7F>@|=mCB6cbj`u(gwV7H?O4EW1 z9u4Yu1m;s%54bk*S~e0w`0PGA<89_lnQtM7<%gyb6R)t$*p|1^{}w`22KB6t00uA6 zf_1o^cqu3knC=fH>;y`q@>C+t5GrQ_@|>uzo#@aoWgCYRzi{9_HkDBU^<2Pa7dUxe zgpURGK2wSbm${h$lrt4d42lr075bLL4QEGrbtOYT$!qGYshy#fVP(&~iJL0FEwCY3 zB_|Nl?y>am@t-${5yrnLz4~Fjf^~O8YSlNN0H<*F(H1#r4cNN#{Zh&fT;f4%Ef5hBy{5R5xUtFeiipsb6%X&g z&~uks*JrWL&wWlMHmGMTtHtSCyDZgzA4;8hoyARH;9y9A0_YI@D+Nd|5YLkh>>P_$BIeKD2lFp&dTIs zAlue76x`{!$J`S(g}FEJi%7qZ16nzXf4>*{b*pan4C%@WF_k)o4^_B4d9c$}Z@oVp z9xCb@3T(1CQP4B3O)Q*?VTx246KLirsu?C-(Os{>L??-zPDpITo;8QfIZhW*_)F|@ zqr=e;PJM!HX0}J$33h(H3b%bH`D?4gHQuD~uUah{t7pRgY1`PcDB0mA^dGF3OWWGz z_V^%PoOZ7!UJx#w6H33evC%**`dEKNd`d2Fs{uG0EbsTPd5FB{t4};D*u@XDiXsS zwLNpC*47sfu#2L-t(hpuB#Mm$S0Ruh-tSn6%Dt~-xuDg&d8uRbglly7uG+1Ht_ZTW z2VJ+jVit!u%-$^e{`FHb0LMhsL0XLrS7jla_o z|L)05hszas5Aa`>J@x)|!~nY>eCuFIEcHRQ@%uJ~nJ+W1HDzwf@1hGzXT}?b&_Gt_ zUd~O;8qn`t6qy~JB_})tsmY6t3KDam(@PMk$zE@1M&OaS_feBdqjg)g1j?L`lGR4Z z#-l`Az*%gCC|Ma^ZCr$upU?ZB z$RYX&@+m+HKNZ(dt4KimnXv{%mf7Aivx+HJNkI#=J8+-p#84~US%1yxG@~u*7MEDC ztDRiK3vte?N+kC4sJ!d=^!0L!yK%I1eXT?H9}khiQV1-_lS0PC7<@^0K0(deJ@5q)dM)}w3YO#j&*OLAG^ zcPWuo^u{wTOAh^D#VpG$Mq@_yHxGldQv~cFXDdMdr#6(D(VGExxMKRX1cI=Q*H;z7 zz%kQFE{R6KU+c+UNjiTYJ~TK9mOBhvf{`}s-RDO|Sp0&qKWyRa`S^L$c>MInPe229 z7SgsH+nt@853}>aV<)7QW%T*fPWHx#C=)fZaAZmGP17n5a5XscP_l4G6bETv9CNOs z=@7OvsrpCDZ$oyDtQ>m`^?9i<@c?SXoeWsshO|8yEB_H(@l$y$^Z=3tp@8eB?)r0dDflu@FOYn>BLM3(_|=5XiR-cHz0-1}cr3$X z#Nzz5sJ%HnDOt>J&@@>-6L5R(2>%y)e+51b*t@_1Pc(N~HtWW=M}Y+w=0 zt$J9g3DOLB0Cx6mmIY}!0K4_9&_TOo90Y#Vos9RyxOYC(n} zKl9rYr_Y`+qVta`&n)T~O3kENBDN1|Pc!(KKvx2ymv zEQ|#p98Q1U#Tgpi$^a4ylG#}xsK(MhF>JjEgwO%4P1om7NbL~F zr^g%Y{txw#uNHa#;$Sw{CzRQFI!GreOke-f1v}W|oEpO1|Ha%ha#!t!@d^E`XvT|; zRUbaOSAT2Z2;BDcA3NQUGpCM4peu4sbu5__K=IPZYo;u!oSk+!-e)*SLN5%H9N__wA z{{MydRkchA>~bx6$AiQ&xEPeh10?8?#l!>rhy3NAPdEO}0>lE_kgI_93OJAlGrFMV z5Vn8U;>>gWfXgViWh$jkIYj-X+Y>-^4H_k8=P8wzc9RO}of`T+9cq|VsGtnONE zw5c@Am?BU`%0*dWBmIk5Mz3de?r{&s-F;QlDHe)el4K-xJjuTLvalGFgi6{!j}}pA zN$`;`2N-W9J7#J+`pE`I-bvwpLKQF{=cA|c-LB6fmI^-ErRZuRT2KBeft%CLpkfQa zA(*crr8@wXwQT&~K#Rs8Rt+{rhh60^$=Ss2Yna9OIr<7VP$FT|%Iy4Ir#TyCyqzwp z+RB0n1KMCKQ(W?k0WTN~6f6FA4LiA+F$4Jksm?9mHOgKON3ou;rVD#e@6#84sDYl0 zZ^W~rU}-Ed9KdP9{S{w-&Co{n6a8E>ra922_P#@OcFk!pb(np7Xt;_w?+w_F^u^?_ zH0;T=z5NKo`LC0fAe=X2O8 z44|EJc*nBJ!efjgLY|gD6e4wy919TeX|dqF)F&>em(N-jhT*SRM}RSDnS!mRqo1F> z$!3o(5E?)Kv~T??K+zT`R%pd5@DXW&ENCG@$Ja;l%*NlXaWdyP@;CZA^w zo>O;aWDvqf0BRJCk6i-%)SL@i{79Z5L#t<^-FRb~RI2WrRH#}=zT|&mk3=6{^LW7t z=+N-}y|kHtzSt^A%IQ0RrQoca37d4hGM`Tq9n&6@KbdrAd_lU*VfxlyMCX!IT9v1s z#kK$?3pL=Y=;yj+3S^m`lRD9B@2H9~bTM0mveMV(>TIB^QSGrvIYXN<{CgiaV%eyU zen#gEOlQQ0nUw)h^OQk%e;@2C%( zB1k|RrvtsdKO3T=H>Jv^|L#16_2uWZEp7e+XPr9sn8Nk|M>C%NdQ=266+cCtH|oX8-R>0(@0ED|gS7#m=fJ=l?AxoHPuPL=5dO2-;GWy+d-l{>mxsLGmu= z1urT-dPLwS4lz7ogi@gyfVUEIEvrUfsHsDGxxN$tqj!U|F!lccCXD&-esvJR4LT{3 zRZ->NBnJvZPz#f82(1B1(um)H?p``@)$`Dzn({u6%ZP*h;WvYxuhXUayC||-h-`Eb z2&jP8A(cp=zCroww(zNZU6SzU0tU^qJ}ub%^9RW#47BXRb7B7e~t9eplcGx`-g{SQ;!0HJz#BOebfI z()mvT2Q^tQ&)jxM$kZQPFWJ$VuGXha4DkLc&KIFY1Ki*VqHE-aKGL`&3m-Bwyz?IS z_+vR$TbutK9l0kW8%!&{{~qMR zJI(BBbg!pUM+=#BqY2CZBU};?h^7fX;la2vskJ*u<37WFP(Ud=_5_+!gt|l%t)W+) z@AO)JC;4q&y}hSxP*FTD?p@U&kY4Z(O-)Sv=+$U{ND$ot_K$#IMc$xb%z@K#P}yi=%I_8K$rRo47MNm#yM$?8zKsK#D)LC)@LSk;mpLC)CaG9>=64;T zHk5>4_+;&#A(28EW|+){lUtow%|jxtAG^f8zN&RRSA4&KqSmKWJPH%xAoqhQYU8t_ zDd^GWr9{xE=4PoQHs4pSgN`cx3@tZ*CP+P}JZ~U=m23ob(Csd4pXJ=Vc`RosOlD&p z4mx0ElP%ayAqK4gQ2Of0R`@}i$WHf9?sze)&PB_Th`M_uyxJp!^QUnNT>#@?1++f* z<-Ms8n{U5+js_o#hY6wEeRUb-;23;p+xGU$ zA~;THTk85oV)h~MOwxhJ!7^db|A@2Bd+Yw>9UfBVndUU|v_^ z<+Y>lFJs(D4+@}hQJrHtJ3BH|oy4!Di`&Gf%716XXRT#Fh;v(Hdq4InPq|n@m9Njv zFEg?eE?G&-9Qb&KGpOz2IQqI}MS+RAlBukiU2hTPyx%l64AKuUJ$+x`^5~>D&^Sw{&O!dDy_GqryZaj20UalB@ zl|3!Yrcf~&FCb;dj}3IDaRHAJ)mD{p)ERTFFIL>}TM-)~v~gZ&quYF9omVU7`V*hg zx1X@~Z__7ti(~?uB$~QE*vgwbIzS~>%T|=d&I|%JO8CBij@WiHc zDWPL|?Kbts4+-Vv!s&jWm)Rj>gR1Ao6>FL@ge8Ae*2a+8wsb)(WU89f@-CzGMe9H~ zmE^*7y_vZ~f1>o|MMft|E_?fCl6vtWrQZw@2#W36e#PUue~*-oh`a81S3lmO%g(4W z*ql%GYNOem!_K)kjGXpk2s9*<`khID!{3!&aGUZ_SD+O3j^w}M5 zI>#$9CkV4W%Lu3=Er_6h>@vNdl zaUXbxdCsX)1c_sh6}La7Ttay!ooLI(j{B0(dt1=!wO~}c8aZX4zKSV*6`^<QegDUXTSG>z^Uc)-|ymGr+!`LC;r`~iQhxg zx0Lzgn}ez6lE@&*XMQ_*UJXI&p{ zbNvGDy&LIYBz|~i6qCuZClD1Tx0#RGbAAOO z(;Z8W0;gY@CwLYJzglU(yFd8OqLu=B7FnhAi-;2cI9N>v{oBG0v`3vn0Z9w)ehXdP zN~v;PE=|qyU+SHS1Cp0U`=;S<#WY$sE{!3W+q3ofR1u{rfopc8b;etSj~2E@){L&p zCITsEzTAIAAM)Sd;HD&*Q2VyBwFry+Xsx7T#r!0kC8NC3^C=+mMk}bA1q9|s zyk;DgSzYGr9WFX=YS3#(gy$WQ2H&d$980C4WYHWeLgm_( z%UHpRh#@d8M{#3@@nO=lB<0V)fQJ~bv|{2%@y?z`guTRTDCxUvR@@o5?H1DeY)!r8 zJ{6_IT9z^2dDw8FyfH_nUXbpIE$bn%T{qGUiB4;Is-`fE7FTBKB$^9gzQoD?UC5m8 z^J4blUFBVfQOXx1#I8z|FGg6HeXp5d6B`_4b?`YtOcmgMsn)muGM_yHDl>$SgUvuMRfWT7Kni!Sfk}|MyFw@F?83g%{ov^d2M2A3o|xY{Q)d+7 zJ#)#4=Uk*+C_XYiHvjO66eqet_sRU;>HMBZuimC^-4&q>z*g1sq>Tg_aCNj{j!oM6 z42dpsa}{dVCvY(kcBmq2sS$epSNoq$kE$%68i2;re&M!r6TB!%`4ph}Os3v*OTE=d zlPiAXy}d|Oj*e^;bwSMv=VvKL^OGqUi|aH$w!3xkddJ*@@#_`NH7}c<8_G_@w4XU^ z>oI9pC9qH0@8=eQS|L@Yiip*B8#32v;g`^1%MQyGA;j+kO|s0Y>AU5sfEmGdL&EM* z1bkEKdQg#@d_b4J55~19`Q1m$0=cj|Q}&9(QM0+bD5Tonz${%z>6jyKrn#85dS=e?#s+MqK({ti8Z7Ot&$$H5&LgZ%j7 zHx;e;a?JTH=?a(?@k+bMTA+BP!4KvLz>~Qg`6yz*PgDXS82jXEO>k`WXpoRM1F7QJ z_RdYm8IOmD$T+OFbMmy7uUA~n#_A)=idCqzk*aB@a9E+~`MKB14Vkbojal!Xm7SZY zU$1EM<4=|TCEr~c!EIz0WOIBD%gDeJ63s1yBaLWK2no}yNWg&=$fozFM*V$N&H85z z@#;h%0$%%JKZ1{q!MlP1C+NZv; z5XOo&KK7=HZ@eeKIQEkRT7l|9t%}E|b;R57W2wsfOsEr&m}i#V14D;NV(m&!Rg->n zSzd*O62zjjR;4TCa)8RicJvV?5ivTP!CwumiwWbAfJac9KqQE+?iy|3hp2U0R5|2N z$cbc72oikN@QnmLYBKIQSFcmIyXSe>bOGWDRa(7$h1q}ebT$dD0&kW{#9l+?@C*cx z#k@m`H>eawLqmHIy6u(@m+1Iz5kcoDN-(aX-ejc}Is@3xZ&5tG#^liHCLhrN$3ExF zxQuCTf-sbF(eR?gG6a zN8R=@l-_B}@0i7drZ7!I7_+OeZjfWa+aZ;+W&D*lD?SziI5ys=!H6Fhzg$pDiLd$V zb#P2G6iu87JaGw@;wl9>`cbY}tCLJofS*}8e-DRo?Q~2EwBwkprSaJY0TWLRlEz_- zaxjT~h&W3dEbCAzd9T6%&h>YK4ttbTr-i>J@~h5BZW9TWR$d`op#~nHoN{M*7F*(H z1sx_YsaEMG*~V!B9soq4{Q1WwBCg@~S0CuW&x}?>+@_LWwVMHqx4QrCp<|crd6+c* zG5&Gol%_%-1*V^1+#hXO@q4?&dr{6_FAL`k7Hp7YM88d%u31ue|J}4C8}(X+2Mih7+4ZzP2VF;f_%tNXdIf#9_CZx}%(kwtPTlHCc0|taf?GlR^D2ed92*2=V~%+^art3U zMVa4UI7Rb9xzWWUB;|cbM`NC;0ggaXnyIX2x(-6LfwQtI@Vf={|NdTpj^u*NVj2?K z9Yi 0$.\n", + "\n", + "A Markov semigroup $(P_t)$ on $S$ is called **irreducible** if $y$ is\n", + "accessible from $x$ for every $(x,y) \\in S \\times S$.\n", + "\n", + "We seek a characterization of irreducibility of $(P_t)$ in terms of its\n", + "generator.\n", + "\n", + "As a first step, we will say there is a **$Q$-positive probability flow** from $x$\n", + "to $y$ if there exists a finite sequence $(z_i)_{i=1}^m$ in $S$ starting at\n", + "$x=z_1$ and ending at $y=z_m$ such that $Q(z_i, z_{i+1}) > 0$ for all $i$.\n", + "\n", + "\n", + "```{proof:lemma}\n", + ":label: equivirr\n", + "\n", + "For distinct states $x$ and $y$, the following statements are equivalent:\n", + "\n", + "1. The state $y$ is accessible from $x$ under $(P_t)$.\n", + "1. There is a positive probability flow from $x$ to $y$ under $Q$.\n", + "```\n", + "\n", + "((prove only for the UC case?))\n", + "\n", + "```{proof:proof}\n", + "For distinct states $x$ and $y$, we have\n", + "\n", + "$$\n", + " P_t(x, y) = t Q(x,y) + \\frac{t^2}{2!} Q^2(x, y) + \\cdots\n", + "$$ (ptexpan)\n", + "\n", + "If $x$ is accessible from $y$, then $P_t(x, y) > 0$ for some $t > 0$, and hence\n", + "$Q^k(x,y) > 0$ for at least one $k \\in \\NN$.\n", + "\n", + "Writing out the matrix product as a sum, we now have\n", + "\n", + "$$\n", + " Q^k(x,y) =\n", + " \\sum_{z_1}\n", + " \\sum_{z_2}\n", + " \\cdots\n", + " \\sum_{z_{k-1}}\n", + " Q(x, z_1) Q(z_1, z_2) \\cdots Q(z_{k-1}, y) \n", + " > 0\n", + "$$ (qkassum)\n", + "\n", + "It follows that at least one element of the sum must be strictly positive, so\n", + "positive probability flow from $x$ to $y$ is established.\n", + "\n", + "To prove the converse, suppose there is positive probability flow from $x$ to\n", + "$y$.\n", + "\n", + "We can then take a finite sequence $(z_i)_{i=1}^m$ starting at $x$ and ending\n", + "at $y$ with $Q(z_i, z_{i+1}) > 0$ for all $i$.\n", + "\n", + "We can and do assume that $(z_i)_{i=0}^m$ is the shortest such sequence\n", + "(otherwise just pick the shortest one).\n", + "\n", + "Because any shorter sequence $(u_i)_{i=1}^k$ linking $x$ and $y$ has $Q(u_i, u_{i+1}) = 0$ for some $i$, {eq}`qkassum` implies that $Q^k(x,y) = 0$ for all $k < m$.\n", + "\n", + "Hence, by {eq}`ptexpan`, we have $P_t(x, y) = t^m Q^m(x, y) + o(t^m)$.\n", + "\n", + "As $Q^m(x, y) > 0$, we see that $P_t(x, y) > 0$ for sufficiently small $t$.\n", + "\n", + "```\n", + "\n", + "\n", + "Irreducible implies aperiodic.\n", + "\n", + "```{proof:theorem}\n", + "For a ((UC)) Markov semigroup $(P_t)$, the following statements are\n", + "equivalent:\n", + "\n", + "1. $(P_t)$ is irreducible.\n", + "1. $P_t(x,y) > 0$ for all $t > 0$ and all $(x,y) \\in S \\times S$.\n", + "```\n", + "\n", + "```{proof:proof}\n", + "\n", + "To be added.\n", + "```\n", + "\n", + "\n", + "Irreducible implies uniqueness.\n", + "\n", + "\n", + "\n", + "## Asymptotic Stabiltiy\n", + "\n", + "To be added.\n", + "\n", + "Include Foguel alternative?\n", + "\n", + "Thm 6.2 of RR and MTK." + ] + } + ], + "metadata": { + "jupytext": { + "formats": "ipynb,md:myst", + "text_representation": { + "extension": ".md", + "format_name": "myst", + "format_version": "0.9", + "jupytext_version": "1.5.0" + } + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.7" + }, + "source_map": [ + 13, + 30, + 40 + ] + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file diff --git a/_sources/ergodicity.md b/_sources/ergodicity.md new file mode 100644 index 0000000..3ce94c5 --- /dev/null +++ b/_sources/ergodicity.md @@ -0,0 +1,213 @@ +--- +jupytext: + formats: ipynb,md:myst + text_representation: + extension: .md + format_name: myst + format_version: '0.9' + jupytext_version: 1.5.0 +kernelspec: + display_name: Python 3 + language: python + name: python3 +--- + + +# Stationarity and Ergodicity + + +## Overview + +To be added. + +Use the distribution flow figures from markov_prop.md, this time starting from +many different distributions. + +Suggests ergodicity. + +We will use the following imports + +```{code-cell} ipython3 +import numpy as np +import scipy as sp +import matplotlib.pyplot as plt +import quantecon as qe +from numba import njit + +from mpl_toolkits.mplot3d import Axes3D +from mpl_toolkits.mplot3d.art3d import Poly3DCollection + +``` + + + +## Stationary Distributions + +Let $(P_t)$ be a Markov semigroup on countable state space $S$. + +A distribution $\psi \in \dD$ is called **stationary** for $(P_t)$ if + +$$ + \psi P_t = \psi + \text{ for all } t \geq 0 +$$ + +In many cases, it is easier to use the generator of the semigroup to identify +stationary distributions. + +The next result makes this possible. + +It is analogous to the idea that a point $\bar x$ in $\RR^d$ is stationary for +a vector ODE $x'_t = F(x_t)$ when $F(\bar x) = 0$. + +It holds true under weaker conditions but the version stated here is easy to +prove and sufficient for most cases we consider. + +```{proof:theorem} +:label: statfromq + +For a conservative intensity matrix $Q$ on $S$ and +associated Markov semigroup $(P_t)$, a distribution $\psi$ on $S$ is stationary +for $(P_t)$ if and only if $\psi Q = 0$. +``` + +```{proof:proof} +Suppose first that $\psi \in \dD$ and $\psi Q = 0$. + +Since $Q$ is conservative, $(P_t)$ is a UC semigroup with $P_t = e^{tQ}$ for +all $t$, and hence, for any $t \geq 0$, + +$$ + \psi e^{tQ} = \psi + t \psi Q + t^2 \frac{\psi Q^2}{2!} + + \cdots +$$ + +From $\psi Q = 0$ we get $\psi Q^k = 0$ for all $k \in \NN$, so the last +display yields $\psi P_t = \psi$. + +Hence $\psi$ is stationary for $(P_t)$. + +Now suppose that $\psi$ is stationary for $(P_t)$ and set $D_t := (1/t) (P_t - +I)$. + +From the triangle inequality and the definition of the operator norm, for any given $t$, + +$$ + \| \psi Q \| + \leq \| \psi (Q - D_t) \| + \| \psi D_t \| + \leq \| Q - D_t \| + \| \psi D_t \| +$$ + +Since $(P_t)$ is UC and $Q$ is its generator, we have $\| D_t - Q \| \to 0$ in +$\lL(\ell_1)$ as $t \to 0$ from the right. + +Hence $\| \psi Q \| \leq \liminf_{t \to 0} \| \psi D_t \|$. + +As $\psi$ is stationary, $\psi D_t = 0$ for all $t$. + +Hence $\psi Q = 0$, as was to be shown. +``` + + + +## Irreducibility and Uniqueness + +Let $(P_t)$ be a Markov semigroup on $S$. + +We say that state $y$ is **accessible** from state $x$ if +there exists a $t \geq 0$ such that $P_t(x, y) > 0$. + +A Markov semigroup $(P_t)$ on $S$ is called **irreducible** if $y$ is +accessible from $x$ for every $(x,y) \in S \times S$. + +We seek a characterization of irreducibility of $(P_t)$ in terms of its +generator. + +As a first step, we will say there is a **$Q$-positive probability flow** from $x$ +to $y$ if there exists a finite sequence $(z_i)_{i=1}^m$ in $S$ starting at +$x=z_1$ and ending at $y=z_m$ such that $Q(z_i, z_{i+1}) > 0$ for all $i$. + + +```{proof:lemma} +:label: equivirr + +For distinct states $x$ and $y$, the following statements are equivalent: + +1. The state $y$ is accessible from $x$ under $(P_t)$. +1. There is a positive probability flow from $x$ to $y$ under $Q$. +``` + +((prove only for the UC case?)) + +```{proof:proof} +For distinct states $x$ and $y$, we have + +$$ + P_t(x, y) = t Q(x,y) + \frac{t^2}{2!} Q^2(x, y) + \cdots +$$ (ptexpan) + +If $x$ is accessible from $y$, then $P_t(x, y) > 0$ for some $t > 0$, and hence +$Q^k(x,y) > 0$ for at least one $k \in \NN$. + +Writing out the matrix product as a sum, we now have + +$$ + Q^k(x,y) = + \sum_{z_1} + \sum_{z_2} + \cdots + \sum_{z_{k-1}} + Q(x, z_1) Q(z_1, z_2) \cdots Q(z_{k-1}, y) + > 0 +$$ (qkassum) + +It follows that at least one element of the sum must be strictly positive, so +positive probability flow from $x$ to $y$ is established. + +To prove the converse, suppose there is positive probability flow from $x$ to +$y$. + +We can then take a finite sequence $(z_i)_{i=1}^m$ starting at $x$ and ending +at $y$ with $Q(z_i, z_{i+1}) > 0$ for all $i$. + +We can and do assume that $(z_i)_{i=0}^m$ is the shortest such sequence +(otherwise just pick the shortest one). + +Because any shorter sequence $(u_i)_{i=1}^k$ linking $x$ and $y$ has $Q(u_i, u_{i+1}) = 0$ for some $i$, {eq}`qkassum` implies that $Q^k(x,y) = 0$ for all $k < m$. + +Hence, by {eq}`ptexpan`, we have $P_t(x, y) = t^m Q^m(x, y) + o(t^m)$. + +As $Q^m(x, y) > 0$, we see that $P_t(x, y) > 0$ for sufficiently small $t$. + +``` + + +Irreducible implies aperiodic. + +```{proof:theorem} +For a ((UC)) Markov semigroup $(P_t)$, the following statements are +equivalent: + +1. $(P_t)$ is irreducible. +1. $P_t(x,y) > 0$ for all $t > 0$ and all $(x,y) \in S \times S$. +``` + +```{proof:proof} + +To be added. +``` + + +Irreducible implies uniqueness. + + + +## Asymptotic Stabiltiy + +To be added. + +Include Foguel alternative? + +Thm 6.2 of RR and MTK. + + diff --git a/_sources/generators.ipynb b/_sources/generators.ipynb new file mode 100644 index 0000000..104fa1b --- /dev/null +++ b/_sources/generators.ipynb @@ -0,0 +1,559 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Semigroups and Generators\n", + "\n", + "\n", + "\n", + "## Overview\n", + "\n", + "We have seen in previous lectures that every intensity matrix generates a\n", + "Markov semigroup.\n", + "\n", + "We have also hinted that the pairing is one-to-one, in a sense to be made precise.\n", + "\n", + "To clarify these ideas, we start in an\n", + "abstract setting, with an arbitrary initial value problem.\n", + "\n", + "In this setting we introduce general operator semigroups and their generators.\n", + "\n", + "Once this is done, we will be able to return to the Markov case and fully\n", + "clarify the connection between intensity matrices and Markov semigroups.\n", + "\n", + "The material below is relatively technical, with most of the\n", + "complications driven by the fact that the state space can be infinite.\n", + "\n", + "Such technicalities are hard to avoid, since so many interesting Markov chains\n", + "do have infinite state spaces.\n", + "\n", + "* Our very first example -- the Poisson process -- has an infinite state space.\n", + "* Another example is the study of queues, which often have no natural upper\n", + "bound.[^footnotepp]\n", + "\n", + "[^footnotepp]: In fact a major concern with queues is that their length does not explode. This issue cannot be properly explored unless the state space is allowed to be infinite.\n", + "\n", + "Readers are assumed to have some basic familiarity with Banach spaces.\n", + "\n", + "## Motivation\n", + "\n", + "The general theory of continuous semigroups of operators is motivated by\n", + "the problem of solving linear ODEs in infinite dimensional spaces.[^fnpde]\n", + "\n", + "[^fnpde]: An excellent introduction to operator semigroups, combined with\n", + " applications to PDEs and Markov processes, can be found in {cite}`applebaum2019semigroups`.\n", + "\n", + "More specifically, the challenge is to solve initial value problems such as \n", + "\n", + "$$\n", + " x'_t = A x_t,\n", + " \\quad x_0 \\text{ given}\n", + "$$ (abscp)\n", + "\n", + "where \n", + "\n", + "* $x_t$ takes value in a Banach space at each time $t$,\n", + "* $A$ is a linear operator and\n", + "* the time derivative $x'_t$ uses a definition appropriate for a Banach\n", + " space.\n", + "\n", + "This problem is also called the \"abstract Cauchy problem\".\n", + "\n", + "Why do we need solve such problems?\n", + "\n", + "One example comes from PDEs. \n", + "\n", + "PDEs tell us how functions change over time, starting from an infinitesimal\n", + "description.\n", + "\n", + "When $x_t$ is a point in a function space, this fits into the framework of\n", + "{eq}`abscp`.\n", + "\n", + "Another example comes from Markov processes, where, as we have seen, the flow\n", + "of distributions over time can be represented as a linear ODE in distribution\n", + "space.\n", + "\n", + "If the number of state is infinite, then the space of distributions is\n", + "infinite dimensional.\n", + "\n", + "This is another version of {eq}`abscp`, and we return to it after a discussion\n", + "of the general theory.\n", + "\n", + "To give a high level view of the results below, the solution to the Cauchy\n", + "problem is represented as a trajectory $t \\mapsto U_t x_0$ from the initial\n", + "value $x_0$ under a semigroup of maps $(U_t)$.\n", + "\n", + "The operator $A$ in {eq}`abscp` is called the \"generator\" of $(U_t)$ and is\n", + "its infinitesimal description.\n", + "\n", + "\n", + "\n", + "## Preliminaries\n", + "\n", + "Throughout this lecture, $(\\BB, \\| \\cdot \\|)$ is a Banach space.\n", + "\n", + "\n", + "### The Space of Linear Operators\n", + "\n", + "You will recall that a **linear operator** on $\\BB$ is a map $A$ from $\\BB$ to\n", + "itself satisfying \n", + "\n", + "$$\n", + " A(\\alpha g + \\beta h) = \\alpha A g + \\beta A h,\n", + " \\quad\n", + " \\forall \\, g, h \\in \\BB, \\;\\; \\alpha, \\beta \\in \\RR\n", + "$$\n", + "\n", + "The operator $A$ is called **bounded** if\n", + "\n", + "$$\n", + " \\| A \\| := \\sup_{g \\in \\BB, \\, \\| g \\| \\leq 1} \\| A g \\| < \\infty\n", + "$$ (norml)\n", + "\n", + "This is the usual definition of a [bounded linear operator](https://en.wikipedia.org/wiki/Bounded_operator) on a normed linear space.\n", + "\n", + "The set of all bounded linear operators on $\\BB$ is denoted by $\\linop$ and is\n", + "itself a Banach space.\n", + "\n", + "\n", + "Sums and scalar products of elements of $\\linop$ are defined in the usual way,\n", + "so that, for $\\alpha \\in \\RR$, $A, B \\in \\linop$ and $g \\in \\BB$, we have \n", + "\n", + "$$\n", + " (A + B) g = Ag + Bg, \n", + " \\quad (\\alpha A) g = \\alpha (A g)\n", + "$$\n", + "\n", + "and so on.\n", + "\n", + "We write $A B$ to indicate composition of the operators $A, B \\in \\linop$.\n", + "\n", + "The value defined in {eq}`norml` is called the **operator norm** of $A$ and,\n", + "as suggested by the notation, [is a norm](https://en.wikipedia.org/wiki/Operator_norm) on $\\linop$.\n", + "\n", + "In addition to being a norm, it enjoys the submultiplicative property $\\| AB\n", + "\\| \\leq \\| A \\| \\| B\\|$ for all $A, B \\in \\linop$.\n", + "\n", + "\n", + "Let $I$ be the identity in $\\linop$, satisfying $Ig = g$ for all $g \\in \\BB$.\n", + "\n", + "(In fact $\\linop$ is a [unital Banach algebra](https://en.wikipedia.org/wiki/Banach_algebra) when multiplication is identified with operator composition and $I$ is adopted as the unit.)\n", + "\n", + "\n", + "\n", + "### The Exponential Function\n", + "\n", + "Given $A \\in \\linop$, the exponential of $A$ is the element of\n", + "$\\linop$ defined as\n", + "\n", + "$$\n", + " e^A \n", + " = \\sum_{k \\geq 0} \\frac{A^k}{k!} \n", + " = I + A + \\frac{A^2}{2!} + \\cdots\n", + "$$ (opexpo)\n", + "\n", + "This is the same as the definition for the matrix exponential.The exponential function arises naturally as the solution to ODEs in Banach\n", + "space, one example of which (as we shall see) is distribution flows\n", + "associated with continuous time Markov chains.\n", + "\n", + "The exponential map has the following properties:\n", + "\n", + "* For each $A \\in \\linop$, the operator $e^A$ is a well defined element of $\\linop$ with $\\| e^A \\| \\leq e^{\\| A \\|}$.[^fncoex]\n", + "* $e^0 = I$, where $0$ is the zero element of $\\linop$.\n", + "* If $A, B \\in \\linop$ and $AB = BA$, then $e^{A + B} = e^A e^B$\n", + "* If $A \\in \\linop$, then $e^A$ is invertible and $(e^A)^{-1} = e^{-A}$.\n", + "\n", + "The last fact is easily checked from the previous ones.\n", + "\n", + "\n", + "[^fncoex]: Convergence of the sum in {eq}`opexpo` follows from boundedness of $A$ and the fact that $\\linop$ is a Banach space.\n", + "\n", + "\n", + "\n", + "### Operator Calculus \n", + "\n", + "Consider a function \n", + "\n", + "$$\n", + " \\RR_+ \\ni t \\mapsto U_t \\in \\linop\n", + "$$\n", + "\n", + "which we can think of as a time path in $\\linop$, such as a flow of Markov\n", + "operators.\n", + "\n", + "We say that this function is **differentiable at $\\tau \\in \\RR_+$** if there exists\n", + "an element $T$ of $\\linop$ such that\n", + "\n", + "$$\n", + " \\frac{U_{\\tau+h} - U_\\tau}{h} \\to T \n", + " \\; \\text{ as } h \\to 0\n", + "$$ (devlim)\n", + "\n", + "In this case, $T$ is called the **derivative** of the function $t \\mapsto U_t$ at $\\tau$ and we write\n", + "\n", + "$$\n", + " T = U'_\\tau \n", + " \\; \\text{ or } \\;\n", + " T = \\frac{d}{dt} U_t \\, \\Big|_{t=\\tau}\n", + "$$\n", + "\n", + "(Convergence of operators is in operator norm. If $\\tau = 0$, then the limit\n", + "$h \\to 0$ in {eq}`devlim` is the right limit.)\n", + "\n", + "```{proof:example}\n", + "If $U_t = t V$ for some fixed $V \\in \\linop$, then it is easy to\n", + "see that $V$ is the derivative of $t \\mapsto U_t$ at every $t \\in \\RR_+$.\n", + "```\n", + "\n", + "```{proof:example}\n", + "In {doc}`our discussion ` of the Kolmogorov forward equation \n", + "when $S$ is finite, we introduced the derivative of a map $t\n", + "\\mapsto P_t$, where each $P_t$ is a matrix on $S$.\n", + "\n", + "The derivative was defined by differentiating $P_t$ element-by-element.\n", + "\n", + "This coincides with the operator-theoretic definition in {eq}`devlim` when $S$\n", + "is finite, because then the space $\\lL(\\ell_1)$ is finite dimensional, and\n", + "hence pointwise and norm convergence coincide.\n", + "```\n", + "\n", + "Analogous to the matrix and scalar cases, we have the following result:\n", + "\n", + "```{proof:lemma} Differentiability of the Exponential Curve\n", + ":label: diffexpmap\n", + "\n", + "For all $A \\in \\linop$, the exponential curve $t \\mapsto e^{tA}$ is everywhere differentiable and \n", + "\n", + "$$\n", + " \\frac{d}{dt} e^{tA} = e^{tA} A = A e^{tA}\n", + "$$ (expdiffer)\n", + "```\n", + "\n", + "The proof is a (solved) exercise (see below).\n", + "\n", + "\n", + "## Semigroups and Generators\n", + "\n", + "For continuous time Markov chains where the state space $S$ is finite, we\n", + "saw that Markov semigroups often take the form $P_t = e^{tQ}$ for some\n", + "intensity matrix $Q$.\n", + "\n", + "This is ideal because the entire semigroup is characterized in a simple way by\n", + "its infinitesimal description $Q$.\n", + "\n", + "It turns out that, when $S$ is finite, this is always true: if $(P_t)$ is a\n", + "Markov semigroup, then there exists an intensity matrix $Q$ satisfying $P_t =\n", + "e^{tQ}$ for all $t$.\n", + "\n", + "Moreover, this statement is again true when $S$ is infinite, provided that\n", + "some restrictions are placed on the semigroup.\n", + "\n", + "Our aim is to make these statements precise, starting in an abstract setting\n", + "and then specializing.\n", + "\n", + "\n", + "### Operator Semigroups\n", + "\n", + "Let $U_t$ be an element of $\\linop$ for all $t \\in \\RR_+$\n", + "\n", + "We say that $(U_t)$ is an **evolution semigroup** on $\\linop$ if $U_0 = I$ and\n", + "$U_{s + t} = U_s U_t$ for all $s, t \\geq 0$.\n", + "\n", + "The idea is that $(U_t)$ generates a path in $\\BB$ from any starting point $g \\in \\BB$, so that $U_t g$ is interpreted as the location of the state after $t$ units of time.\n", + "\n", + "An evolution semigroup $(U_t)$ is called \n", + "\n", + "* a $C_0$ **semigroup** on $\\BB$ if, for each $g \\in \\BB$, the map $t \\mapsto U_t g$ from $\\RR_+$ to $\\BB$ is continuous, and\n", + "* a **uniformly continuous semigroup** on $\\BB$ if the map $t \\mapsto U_t$ from $\\RR_+$ to $\\linop$ is continuous.\n", + "\n", + "In what follows we abbreviate \"uniformly continuous\" to UC.[^ucnote]\n", + "\n", + "[^ucnote]: Be careful: the definition of a UC semigroup requires that \n", + "$t \\mapsto U_t$ is continuous as a map into $\\linop$, rather than uniformly\n", + "continuous. The UC terminology comes about because, for a UC semigroup, we\n", + "have, by definition of the operator norm,\n", + "$\\sup_{\\| g \\| \\leq 1} \\| U_s g - U_t g \\| \\to 0$ when $s \\to t$.\n", + "\n", + "\n", + "```{proof:example} Exponential curves are UC semigroups\n", + ":label: ecuc\n", + "\n", + "If $U_t = e^{tA}$ for $t \\in \\RR_+$ and $A \\in \\linop$, then $(U_t)$\n", + "is a uniformly continuous semigroup on $\\BB$.\n", + "```\n", + "\n", + "The claim that $(U_t)$ is an evolution semigroup follows directly from the\n", + "properties of the exponential function given above.\n", + "\n", + "Uniform continuity can be established using arguments similar to those in the\n", + "proof of differentiability in {proof:ref}`diffexpmap`. \n", + "\n", + "Since norm convergence on $\\linop$ implies pointwise convergence, every\n", + "uniformly continuous semigroup is a $C_0$ semigroup.\n", + "\n", + "The reverse is certainly not true --- there are many important $C_0$ semigroups that fail to be uniformly continuous.\n", + "\n", + "In fact semigroups associated with PDEs, diffusions and other Markov processes\n", + "on continuous state spaces are typically $C_0$ but not uniformly continuous.\n", + "\n", + "There are also important examples of Markov semigroups on infinite\n", + "discrete state spaces that fail to be uniformly continuous.\n", + "\n", + "However, we will soon see that, for most continuous time Markov chains used in\n", + "applications, the semigroups are uniformly continuous.\n", + "\n", + "\n", + "\n", + "\n", + "### Generators\n", + "\n", + "Consider a continuous time Markov chain on a finite state space with intensity\n", + "matrix $Q$.\n", + "\n", + "The Markov semigroup $(P_t)$ is fully specified by this infinitesimal\n", + "description $Q$, in the sense that \n", + "\n", + "* $P_t = e^{tQ}$ for all $t \\geq 0$ and (equivalently)\n", + "* the forward and backward equations hold: $P_t' = P_t Q = Q P_t$.\n", + "\n", + "Since $P_0 = I$, the matrix $Q$ can be recovered from the semigroup via \n", + "\n", + "$$\n", + " Q = P'_0 = \\lim_{h \\downarrow 0} \\frac{P_h - I}{h} \n", + "$$\n", + "\n", + "In the more abstract setting of $C_0$ semigroups, we say that $Q$ is the\n", + "\"generator\" of the semigroup $P_t$.\n", + "\n", + "More generally, given a $C_0$ semigroup $(U_t)$, we say that a linear\n", + "operator $A$ from $\\BB$ to itself is the **generator** of $(U_t)$ if \n", + "\n", + "$$\n", + " A g = \\lim_{h \\downarrow 0} \\frac{U_h g - g}{h} \n", + "$$ (defgenr)\n", + "\n", + "for all $g \\in \\BB$ such that the limit exists. \n", + "\n", + "The set of points where the limit exists (the domain of the generator) is\n", + "denoted by $D(A)$.\n", + "\n", + "At this point we would like to write {eq}`defgenr` as $A = U'_0$, or express\n", + "$U_t$ as $e^{tA}$, analogous to the Markov case.\n", + "\n", + "There are problems, however.\n", + "\n", + "One problem is that the limit in {eq}`defgenr` can fail to exist for some $g\n", + "\\in \\BB$.\n", + "\n", + "Indeed, why should the limit exist, given that $C_0$ semigroups are not\n", + "required to be differentiable?\n", + "\n", + "The other problem is that, even though the limit exists, the linear operator\n", + "$A$ is not bounded (i.e., not an element of $\\linop$), so \n", + "a statement like $U_t = e^{tA}$ is problematic.\n", + "\n", + "It turns out that, despite these issues, the theory of $C_0$ semigroups is\n", + "powerful, and, with some work, the technical issues can be\n", + "circumvented.[^fnhy]\n", + "\n", + "[^fnhy]: An excellent treatment of the general theory of $C_0$ semigroups, can be found in {cite}`bobrowski2005functional`.\n", + "\n", + "Even better, for the applications we wish to consider, we can focus on UC\n", + "semigroups, where these problems do not arise.\n", + "\n", + "The next section gives details.\n", + "\n", + "\n", + "### A Characterization of Uniformly Continuous Semigroups\n", + "\n", + "We saw in {proof:ref}`ecuc` that exponential curves are an example\n", + "of a UC semigroup.\n", + "\n", + "The next theorem tells us that there are no other examples.\n", + "\n", + "```{proof:theorem} UC Semigroups are Exponential Curves\n", + ":label: ucsgec\n", + "\n", + "If $(U_t)$ is a UC semigroup on $\\BB$, then there exists an $A \\in \\linop$\n", + "such that $U_t = e^{tA}$ for all $t \\geq 0$. Moreover,\n", + "\n", + "* $U_t$ is differentiable at every $t \\geq 0$,\n", + "* $A$ is the generator of $(U_t)$ and\n", + "* $U_t' = A U_t = U_t A$ for all $t \\geq 0$.\n", + "```\n", + "\n", + "The last three claims in {proof:ref}`ucsgec` follow directly from the\n", + "first claim.\n", + "\n", + "The statement $U_t' = A U_t = U_t A$ is a\n", + "generalization of the Kolmogorov forward and backward equations.\n", + "\n", + "While slightly more complicated in the Banach setting, the proof of the first\n", + "claim (existence of an exponential representation) is a direct extension of\n", + "the fact that any continuous function $f$ from $\\RR$ to itself\n", + "satisfying \n", + "\n", + "* $f(s)f(t) = f(s+t)$ for all $s,t \\geq 0$ and\n", + "* $f(0) = 1$ \n", + "\n", + "also satisfies $f(t) = e^{ta}$ for some $a \\in \\RR$.\n", + "\n", + "We proved something quite similar in {proof:ref}`exp_unique`, on \n", + "the memoryless property of the exponential function.\n", + "\n", + "For more discussion of the scalar case, see, for example,\n", + "{cite}`sahoo2011introduction`. \n", + "\n", + "For a full proof of the first claim in {proof:ref}`ucsgec`, in the setting of\n", + "a Banach algebra, see, for\n", + "example, Chapter 7 of {cite}`bobrowski2005functional`.\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "## Exercises\n", + "\n", + "### Exercise 1\n", + "\n", + "Prove that {eq}`expdiffer` holds for all $A \\in \\linop$.\n", + "\n", + "### Exercise 2\n", + "\n", + "In many texts, a $C_0$ semigroup is defined as an evolution semigroup $(U_t)$\n", + "such that \n", + "\n", + "$$\n", + " U_t g \\to g \\text{ as } t \\to 0 \\text{ for any } g \\in \\BB\n", + "$$ (czsg2)\n", + "\n", + "Our aim is to show that {eq}`czsg2` implies continuity at every point $t$, as\n", + "in the definition we used above.\n", + "\n", + "The [Banach--Steinhaus Theorem](https://en.wikipedia.org/wiki/Uniform_boundedness_principle) can be used to show that, for an evolution semigroup $(U_t)$ satisfying {eq}`czsg2`, there exist finite constants $\\omega$ and $M$ such that\n", + "\n", + "$$ \n", + " \\| U_t \\| \\leq e^{t\\omega} M \n", + " \\quad \\text{for all } \\; t \\geq 0\n", + "$$ (sgbound)\n", + "\n", + "Using this and {eq}`czsg2`, show that, for any $g \\in \\BB$, the map $t \\mapsto\n", + "U_t g$ is continuous at all $t$.\n", + "\n", + "\n", + "### Exercise 3\n", + "\n", + "Following on from the previous exercise, \n", + "a UC semigroup is often defined as an evolution semigroup $(U_t)$\n", + "such that \n", + "\n", + "$$\n", + " \\| U_t - I \\| \\to 0 \\text{ as } t \\to 0 \n", + "$$ (czsg3)\n", + "\n", + "Show that {eq}`czsg3` implies norm continuity at every point $t$, as\n", + "in the definition we used above.\n", + "\n", + "In particular, show that, for any $t_n \\to t$, we have\n", + "$\\| U_{t_n} - U_t \\| \\to 0$ as $n \\to \\infty$.\n", + "\n", + "\n", + "\n", + "## Solutions\n", + "\n", + "### Solution to Exercise 1\n", + "\n", + "To show the first equality, fix $t \\in \\RR_+$, take $h > 0$ and observe that\n", + "\n", + "$$\n", + " e^{(t+h)A} - e^{tA} - e^{tA} A\n", + " = e^{tA} (e^{hA} - I - A)\n", + "$$\n", + "\n", + "Since the norm on $\\linop$ is submultiplicative, it suffices to show that \n", + "$\\| e^{hA} - I - A \\| = o(h)$ as $h \\to 0$.\n", + "\n", + "Using the definition of the exponential, this is easily verified,\n", + "completing the proof of the first equality in {eq}`expdiffer`.\n", + "\n", + "The proof of the second equality is similar.\n", + "\n", + "\n", + "### Solution to Exercise 2\n", + "\n", + "Let $(U_t)$ be an evolution semigroup satisfying {eq}`czsg2` and let\n", + "$\\omega$ and $M$ be as in {eq}`sgbound`.\n", + "\n", + "Pick any $g \\in \\BB$, $t > 0$ and $h_n \\downarrow 0$. \n", + "\n", + "On one hand, $U_{t+ h_n} g = U_{h_n} U_t g \\to U_t g$ by {eq}`czsg2`.\n", + "\n", + "On the other hand, from {eq}`sgbound` and the definition of the operator norm,\n", + "\n", + "$$\n", + " \\| U_{t-h_n} g - U_t g\\|\n", + " = \\| U_{t-h_n} ( g - U_{h_n} g) \\|\n", + " \\leq e^{(t-h_n)\\omega} M \\| g - U_{h_n} g\\|\n", + " \\to 0\n", + "$$\n", + "\n", + "as $n \\to \\infty$. This completes the proof.\n", + "\n", + "### Solution to Exercise 3\n", + "\n", + "The solution is similar to that of the previous exercise.\n", + "\n", + "Let $(U_t)$ be an evolution semigroup satisfying {eq}`czsg3`,\n", + "fix $t > 0$ and take $(h_n)$ to be a scalar sequence satisfying $h_n \\downarrow 0$. \n", + "\n", + "On one hand, $U_{t+ h_n} = U_{h_n} U_t \\to U_t $ by {eq}`czsg3`.\n", + "\n", + "On the other hand, from the submultiplicative property of the operator norm\n", + "and {eq}`sgbound`,\n", + "\n", + "$$\n", + " \\| U_{t-h_n} - U_t \\|\n", + " = \\| U_{t-h_n} ( I - U_{h_n}) \\|\n", + " \\leq e^{(t-h_n)\\omega} M \\| I - U_{h_n} \\|\n", + "$$\n", + "\n", + "This converges to 0 as $n \\to \\infty$, completing our proof." + ] + } + ], + "metadata": { + "jupytext": { + "formats": "ipynb,md:myst", + "text_representation": { + "extension": ".md", + "format_name": "myst", + "format_version": "0.9", + "jupytext_version": "1.5.0" + } + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.7" + }, + "source_map": [ + 13 + ] + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file diff --git a/_sources/generators.md b/_sources/generators.md new file mode 100644 index 0000000..143cd37 --- /dev/null +++ b/_sources/generators.md @@ -0,0 +1,530 @@ +--- +jupytext: + formats: ipynb,md:myst + text_representation: + extension: .md + format_name: myst + format_version: '0.9' + jupytext_version: 1.5.0 +kernelspec: + display_name: Python 3 + language: python + name: python3 +--- + +# Semigroups and Generators + + + +## Overview + +We have seen in previous lectures that every intensity matrix generates a +Markov semigroup. + +We have also hinted that the pairing is one-to-one, in a sense to be made precise. + +To clarify these ideas, we start in an +abstract setting, with an arbitrary initial value problem. + +In this setting we introduce general operator semigroups and their generators. + +Once this is done, we will be able to return to the Markov case and fully +clarify the connection between intensity matrices and Markov semigroups. + +The material below is relatively technical, with most of the +complications driven by the fact that the state space can be infinite. + +Such technicalities are hard to avoid, since so many interesting Markov chains +do have infinite state spaces. + +* Our very first example -- the Poisson process -- has an infinite state space. +* Another example is the study of queues, which often have no natural upper +bound.[^footnotepp] + +[^footnotepp]: In fact a major concern with queues is that their length does not explode. This issue cannot be properly explored unless the state space is allowed to be infinite. + +Readers are assumed to have some basic familiarity with Banach spaces. + +## Motivation + +The general theory of continuous semigroups of operators is motivated by +the problem of solving linear ODEs in infinite dimensional spaces.[^fnpde] + +[^fnpde]: An excellent introduction to operator semigroups, combined with + applications to PDEs and Markov processes, can be found in {cite}`applebaum2019semigroups`. + +More specifically, the challenge is to solve initial value problems such as + +$$ + x'_t = A x_t, + \quad x_0 \text{ given} +$$ (abscp) + +where + +* $x_t$ takes value in a Banach space at each time $t$, +* $A$ is a linear operator and +* the time derivative $x'_t$ uses a definition appropriate for a Banach + space. + +This problem is also called the "abstract Cauchy problem". + +Why do we need solve such problems? + +One example comes from PDEs. + +PDEs tell us how functions change over time, starting from an infinitesimal +description. + +When $x_t$ is a point in a function space, this fits into the framework of +{eq}`abscp`. + +Another example comes from Markov processes, where, as we have seen, the flow +of distributions over time can be represented as a linear ODE in distribution +space. + +If the number of state is infinite, then the space of distributions is +infinite dimensional. + +This is another version of {eq}`abscp`, and we return to it after a discussion +of the general theory. + +To give a high level view of the results below, the solution to the Cauchy +problem is represented as a trajectory $t \mapsto U_t x_0$ from the initial +value $x_0$ under a semigroup of maps $(U_t)$. + +The operator $A$ in {eq}`abscp` is called the "generator" of $(U_t)$ and is +its infinitesimal description. + + + +## Preliminaries + +Throughout this lecture, $(\BB, \| \cdot \|)$ is a Banach space. + + +### The Space of Linear Operators + +You will recall that a **linear operator** on $\BB$ is a map $A$ from $\BB$ to +itself satisfying + +$$ + A(\alpha g + \beta h) = \alpha A g + \beta A h, + \quad + \forall \, g, h \in \BB, \;\; \alpha, \beta \in \RR +$$ + +The operator $A$ is called **bounded** if + +$$ + \| A \| := \sup_{g \in \BB, \, \| g \| \leq 1} \| A g \| < \infty +$$ (norml) + +This is the usual definition of a [bounded linear operator](https://en.wikipedia.org/wiki/Bounded_operator) on a normed linear space. + +The set of all bounded linear operators on $\BB$ is denoted by $\linop$ and is +itself a Banach space. + + +Sums and scalar products of elements of $\linop$ are defined in the usual way, +so that, for $\alpha \in \RR$, $A, B \in \linop$ and $g \in \BB$, we have + +$$ + (A + B) g = Ag + Bg, + \quad (\alpha A) g = \alpha (A g) +$$ + +and so on. + +We write $A B$ to indicate composition of the operators $A, B \in \linop$. + +The value defined in {eq}`norml` is called the **operator norm** of $A$ and, +as suggested by the notation, [is a norm](https://en.wikipedia.org/wiki/Operator_norm) on $\linop$. + +In addition to being a norm, it enjoys the submultiplicative property $\| AB +\| \leq \| A \| \| B\|$ for all $A, B \in \linop$. + + +Let $I$ be the identity in $\linop$, satisfying $Ig = g$ for all $g \in \BB$. + +(In fact $\linop$ is a [unital Banach algebra](https://en.wikipedia.org/wiki/Banach_algebra) when multiplication is identified with operator composition and $I$ is adopted as the unit.) + + + +### The Exponential Function + +Given $A \in \linop$, the exponential of $A$ is the element of +$\linop$ defined as + +$$ + e^A + = \sum_{k \geq 0} \frac{A^k}{k!} + = I + A + \frac{A^2}{2!} + \cdots +$$ (opexpo) + +This is the same as the definition for the matrix exponential.The exponential function arises naturally as the solution to ODEs in Banach +space, one example of which (as we shall see) is distribution flows +associated with continuous time Markov chains. + +The exponential map has the following properties: + +* For each $A \in \linop$, the operator $e^A$ is a well defined element of $\linop$ with $\| e^A \| \leq e^{\| A \|}$.[^fncoex] +* $e^0 = I$, where $0$ is the zero element of $\linop$. +* If $A, B \in \linop$ and $AB = BA$, then $e^{A + B} = e^A e^B$ +* If $A \in \linop$, then $e^A$ is invertible and $(e^A)^{-1} = e^{-A}$. + +The last fact is easily checked from the previous ones. + + +[^fncoex]: Convergence of the sum in {eq}`opexpo` follows from boundedness of $A$ and the fact that $\linop$ is a Banach space. + + + +### Operator Calculus + +Consider a function + +$$ + \RR_+ \ni t \mapsto U_t \in \linop +$$ + +which we can think of as a time path in $\linop$, such as a flow of Markov +operators. + +We say that this function is **differentiable at $\tau \in \RR_+$** if there exists +an element $T$ of $\linop$ such that + +$$ + \frac{U_{\tau+h} - U_\tau}{h} \to T + \; \text{ as } h \to 0 +$$ (devlim) + +In this case, $T$ is called the **derivative** of the function $t \mapsto U_t$ at $\tau$ and we write + +$$ + T = U'_\tau + \; \text{ or } \; + T = \frac{d}{dt} U_t \, \Big|_{t=\tau} +$$ + +(Convergence of operators is in operator norm. If $\tau = 0$, then the limit +$h \to 0$ in {eq}`devlim` is the right limit.) + +```{proof:example} +If $U_t = t V$ for some fixed $V \in \linop$, then it is easy to +see that $V$ is the derivative of $t \mapsto U_t$ at every $t \in \RR_+$. +``` + +```{proof:example} +In {doc}`our discussion ` of the Kolmogorov forward equation +when $S$ is finite, we introduced the derivative of a map $t +\mapsto P_t$, where each $P_t$ is a matrix on $S$. + +The derivative was defined by differentiating $P_t$ element-by-element. + +This coincides with the operator-theoretic definition in {eq}`devlim` when $S$ +is finite, because then the space $\lL(\ell_1)$ is finite dimensional, and +hence pointwise and norm convergence coincide. +``` + +Analogous to the matrix and scalar cases, we have the following result: + +```{proof:lemma} Differentiability of the Exponential Curve +:label: diffexpmap + +For all $A \in \linop$, the exponential curve $t \mapsto e^{tA}$ is everywhere differentiable and + +$$ + \frac{d}{dt} e^{tA} = e^{tA} A = A e^{tA} +$$ (expdiffer) +``` + +The proof is a (solved) exercise (see below). + + +## Semigroups and Generators + +For continuous time Markov chains where the state space $S$ is finite, we +saw that Markov semigroups often take the form $P_t = e^{tQ}$ for some +intensity matrix $Q$. + +This is ideal because the entire semigroup is characterized in a simple way by +its infinitesimal description $Q$. + +It turns out that, when $S$ is finite, this is always true: if $(P_t)$ is a +Markov semigroup, then there exists an intensity matrix $Q$ satisfying $P_t = +e^{tQ}$ for all $t$. + +Moreover, this statement is again true when $S$ is infinite, provided that +some restrictions are placed on the semigroup. + +Our aim is to make these statements precise, starting in an abstract setting +and then specializing. + + +### Operator Semigroups + +Let $U_t$ be an element of $\linop$ for all $t \in \RR_+$ + +We say that $(U_t)$ is an **evolution semigroup** on $\linop$ if $U_0 = I$ and +$U_{s + t} = U_s U_t$ for all $s, t \geq 0$. + +The idea is that $(U_t)$ generates a path in $\BB$ from any starting point $g \in \BB$, so that $U_t g$ is interpreted as the location of the state after $t$ units of time. + +An evolution semigroup $(U_t)$ is called + +* a $C_0$ **semigroup** on $\BB$ if, for each $g \in \BB$, the map $t \mapsto U_t g$ from $\RR_+$ to $\BB$ is continuous, and +* a **uniformly continuous semigroup** on $\BB$ if the map $t \mapsto U_t$ from $\RR_+$ to $\linop$ is continuous. + +In what follows we abbreviate "uniformly continuous" to UC.[^ucnote] + +[^ucnote]: Be careful: the definition of a UC semigroup requires that +$t \mapsto U_t$ is continuous as a map into $\linop$, rather than uniformly +continuous. The UC terminology comes about because, for a UC semigroup, we +have, by definition of the operator norm, +$\sup_{\| g \| \leq 1} \| U_s g - U_t g \| \to 0$ when $s \to t$. + + +```{proof:example} Exponential curves are UC semigroups +:label: ecuc + +If $U_t = e^{tA}$ for $t \in \RR_+$ and $A \in \linop$, then $(U_t)$ +is a uniformly continuous semigroup on $\BB$. +``` + +The claim that $(U_t)$ is an evolution semigroup follows directly from the +properties of the exponential function given above. + +Uniform continuity can be established using arguments similar to those in the +proof of differentiability in {proof:ref}`diffexpmap`. + +Since norm convergence on $\linop$ implies pointwise convergence, every +uniformly continuous semigroup is a $C_0$ semigroup. + +The reverse is certainly not true --- there are many important $C_0$ semigroups that fail to be uniformly continuous. + +In fact semigroups associated with PDEs, diffusions and other Markov processes +on continuous state spaces are typically $C_0$ but not uniformly continuous. + +There are also important examples of Markov semigroups on infinite +discrete state spaces that fail to be uniformly continuous. + +However, we will soon see that, for most continuous time Markov chains used in +applications, the semigroups are uniformly continuous. + + + + +### Generators + +Consider a continuous time Markov chain on a finite state space with intensity +matrix $Q$. + +The Markov semigroup $(P_t)$ is fully specified by this infinitesimal +description $Q$, in the sense that + +* $P_t = e^{tQ}$ for all $t \geq 0$ and (equivalently) +* the forward and backward equations hold: $P_t' = P_t Q = Q P_t$. + +Since $P_0 = I$, the matrix $Q$ can be recovered from the semigroup via + +$$ + Q = P'_0 = \lim_{h \downarrow 0} \frac{P_h - I}{h} +$$ + +In the more abstract setting of $C_0$ semigroups, we say that $Q$ is the +"generator" of the semigroup $P_t$. + +More generally, given a $C_0$ semigroup $(U_t)$, we say that a linear +operator $A$ from $\BB$ to itself is the **generator** of $(U_t)$ if + +$$ + A g = \lim_{h \downarrow 0} \frac{U_h g - g}{h} +$$ (defgenr) + +for all $g \in \BB$ such that the limit exists. + +The set of points where the limit exists (the domain of the generator) is +denoted by $D(A)$. + +At this point we would like to write {eq}`defgenr` as $A = U'_0$, or express +$U_t$ as $e^{tA}$, analogous to the Markov case. + +There are problems, however. + +One problem is that the limit in {eq}`defgenr` can fail to exist for some $g +\in \BB$. + +Indeed, why should the limit exist, given that $C_0$ semigroups are not +required to be differentiable? + +The other problem is that, even though the limit exists, the linear operator +$A$ is not bounded (i.e., not an element of $\linop$), so +a statement like $U_t = e^{tA}$ is problematic. + +It turns out that, despite these issues, the theory of $C_0$ semigroups is +powerful, and, with some work, the technical issues can be +circumvented.[^fnhy] + +[^fnhy]: An excellent treatment of the general theory of $C_0$ semigroups, can be found in {cite}`bobrowski2005functional`. + +Even better, for the applications we wish to consider, we can focus on UC +semigroups, where these problems do not arise. + +The next section gives details. + + +### A Characterization of Uniformly Continuous Semigroups + +We saw in {proof:ref}`ecuc` that exponential curves are an example +of a UC semigroup. + +The next theorem tells us that there are no other examples. + +```{proof:theorem} UC Semigroups are Exponential Curves +:label: ucsgec + +If $(U_t)$ is a UC semigroup on $\BB$, then there exists an $A \in \linop$ +such that $U_t = e^{tA}$ for all $t \geq 0$. Moreover, + +* $U_t$ is differentiable at every $t \geq 0$, +* $A$ is the generator of $(U_t)$ and +* $U_t' = A U_t = U_t A$ for all $t \geq 0$. +``` + +The last three claims in {proof:ref}`ucsgec` follow directly from the +first claim. + +The statement $U_t' = A U_t = U_t A$ is a +generalization of the Kolmogorov forward and backward equations. + +While slightly more complicated in the Banach setting, the proof of the first +claim (existence of an exponential representation) is a direct extension of +the fact that any continuous function $f$ from $\RR$ to itself +satisfying + +* $f(s)f(t) = f(s+t)$ for all $s,t \geq 0$ and +* $f(0) = 1$ + +also satisfies $f(t) = e^{ta}$ for some $a \in \RR$. + +We proved something quite similar in {proof:ref}`exp_unique`, on +the memoryless property of the exponential function. + +For more discussion of the scalar case, see, for example, +{cite}`sahoo2011introduction`. + +For a full proof of the first claim in {proof:ref}`ucsgec`, in the setting of +a Banach algebra, see, for +example, Chapter 7 of {cite}`bobrowski2005functional`. + + + + + +## Exercises + +### Exercise 1 + +Prove that {eq}`expdiffer` holds for all $A \in \linop$. + +### Exercise 2 + +In many texts, a $C_0$ semigroup is defined as an evolution semigroup $(U_t)$ +such that + +$$ + U_t g \to g \text{ as } t \to 0 \text{ for any } g \in \BB +$$ (czsg2) + +Our aim is to show that {eq}`czsg2` implies continuity at every point $t$, as +in the definition we used above. + +The [Banach--Steinhaus Theorem](https://en.wikipedia.org/wiki/Uniform_boundedness_principle) can be used to show that, for an evolution semigroup $(U_t)$ satisfying {eq}`czsg2`, there exist finite constants $\omega$ and $M$ such that + +$$ + \| U_t \| \leq e^{t\omega} M + \quad \text{for all } \; t \geq 0 +$$ (sgbound) + +Using this and {eq}`czsg2`, show that, for any $g \in \BB$, the map $t \mapsto +U_t g$ is continuous at all $t$. + + +### Exercise 3 + +Following on from the previous exercise, +a UC semigroup is often defined as an evolution semigroup $(U_t)$ +such that + +$$ + \| U_t - I \| \to 0 \text{ as } t \to 0 +$$ (czsg3) + +Show that {eq}`czsg3` implies norm continuity at every point $t$, as +in the definition we used above. + +In particular, show that, for any $t_n \to t$, we have +$\| U_{t_n} - U_t \| \to 0$ as $n \to \infty$. + + + +## Solutions + +### Solution to Exercise 1 + +To show the first equality, fix $t \in \RR_+$, take $h > 0$ and observe that + +$$ + e^{(t+h)A} - e^{tA} - e^{tA} A + = e^{tA} (e^{hA} - I - A) +$$ + +Since the norm on $\linop$ is submultiplicative, it suffices to show that +$\| e^{hA} - I - A \| = o(h)$ as $h \to 0$. + +Using the definition of the exponential, this is easily verified, +completing the proof of the first equality in {eq}`expdiffer`. + +The proof of the second equality is similar. + + +### Solution to Exercise 2 + +Let $(U_t)$ be an evolution semigroup satisfying {eq}`czsg2` and let +$\omega$ and $M$ be as in {eq}`sgbound`. + +Pick any $g \in \BB$, $t > 0$ and $h_n \downarrow 0$. + +On one hand, $U_{t+ h_n} g = U_{h_n} U_t g \to U_t g$ by {eq}`czsg2`. + +On the other hand, from {eq}`sgbound` and the definition of the operator norm, + +$$ + \| U_{t-h_n} g - U_t g\| + = \| U_{t-h_n} ( g - U_{h_n} g) \| + \leq e^{(t-h_n)\omega} M \| g - U_{h_n} g\| + \to 0 +$$ + +as $n \to \infty$. This completes the proof. + +### Solution to Exercise 3 + +The solution is similar to that of the previous exercise. + +Let $(U_t)$ be an evolution semigroup satisfying {eq}`czsg3`, +fix $t > 0$ and take $(h_n)$ to be a scalar sequence satisfying $h_n \downarrow 0$. + +On one hand, $U_{t+ h_n} = U_{h_n} U_t \to U_t $ by {eq}`czsg3`. + +On the other hand, from the submultiplicative property of the operator norm +and {eq}`sgbound`, + +$$ + \| U_{t-h_n} - U_t \| + = \| U_{t-h_n} ( I - U_{h_n}) \| + \leq e^{(t-h_n)\omega} M \| I - U_{h_n} \| +$$ + +This converges to 0 as $n \to \infty$, completing our proof. diff --git a/_sources/intro.md b/_sources/intro.md new file mode 100644 index 0000000..1945286 --- /dev/null +++ b/_sources/intro.md @@ -0,0 +1,7 @@ +Continuous Time Markov Chains +============================= + +Introductory text to be added. + +Uses --- list applications. + diff --git a/_sources/kolmogorov_bwd.ipynb b/_sources/kolmogorov_bwd.ipynb new file mode 100644 index 0000000..06f5b55 --- /dev/null +++ b/_sources/kolmogorov_bwd.ipynb @@ -0,0 +1,773 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Kolmogorov Backward Equation\n", + "\n", + "## Overview \n", + "\n", + "As models become more complex, deriving analytical representations of the\n", + "Markov semigroup $(P_t)$ becomes harder.\n", + "\n", + "This is analogous to the idea that solutions to continuous time models often\n", + "lack analytical solutions.\n", + "\n", + "For example, when studying deterministic paths in continuous time,\n", + "infinitesimal descriptions (ODEs and PDEs) are often more intuitive and easier\n", + "to write down than the associated solutions.\n", + "\n", + "(This is one of the shining insights of mathematics, beginning with the work\n", + "of great scientists such as Isaac Newton.)\n", + "\n", + "We will see in this lecture that the same is true for continuous time Markov\n", + "chains.\n", + "\n", + "To help us focus on intuition in this lecture, rather than technicalities, the state space is assumed to be finite, with $|S|=n$.\n", + "\n", + "Later we will investigate the case where $|S| = \\infty$.\n", + "\n", + "We will use the following imports" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import scipy as sp\n", + "import matplotlib.pyplot as plt\n", + "import quantecon as qe\n", + "from numba import njit\n", + "\n", + "from scipy.linalg import expm\n", + "from scipy.stats import binom" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "(sdji)=\n", + "## State Dependent Jump Intensities\n", + "\n", + "As we have seen, continuous time Markov chains jump between states, and hence can \n", + "have the form\n", + "\n", + "$$\n", + " X_t = \\sum_{k \\geq 0} Y_k \\mathbb 1\\{J_k \\leq t < J_{k+1}\\}\n", + " \\qquad (t \\geq 0)\n", + "$$ \n", + "\n", + "where $(J_k)$ are jump times and $(Y_k)$ are the states at each jump.\n", + "\n", + "(We are assuming that $J_k \\to \\infty$ with probability one, so that $X_t$ is well\n", + "defined for all $t \\geq 0$, but this is always true for when holding times are exponential and the state space is finite.)\n", + "\n", + "In the {doc}`previous lecture `, \n", + "\n", + "* the sequence $(Y_k)$ was drawn from a Markov matrix $K$ and called the embedded jump chain, while\n", + "* the holding times $W_k := J_k - J_{k-1}$ were IID and Exp$(\\lambda)$ for some\n", + "constant jump intensity $\\lambda$.\n", + "\n", + "In this lecture, we will generalize by allowing the jump intensity to vary\n", + "with the state.\n", + "\n", + "This difference sounds minor but in fact it will allow us to reach full generality\n", + "in our description of continuous time Markov chains, as\n", + "clarified below.\n", + "\n", + "### Motivation\n", + "\n", + "As a motivating example, recall {ref}`the inventory model `,\n", + "where we assumed that the wait time for the next customer was equal\n", + "to the wait time for new inventory.\n", + "\n", + "This assumption was made purely for convenience and seems unlikely to hold true.\n", + "\n", + "When we relax it, the jump intensities depend on the state.\n", + "\n", + "(jumpchainalgo)=\n", + "### Embedded Jump Chain Algorithm\n", + "\n", + "We start with three primitives\n", + "\n", + "1. An initial condition $\\psi$, \n", + "1. a Markov matrix $K$ on $S$ satisfying $K(x, x) = 0$ for all $x \\in S$\n", + " and\n", + "1. a function $\\lambda$ mapping $S$ to $(0, \\infty)$.\n", + "\n", + "The process $(X_t)$ \n", + "\n", + "* starts at state $x$, draw from $\\psi$,\n", + "* waits there for an exponential time $W$ with rate $\\lambda(x)$ and then\n", + "* updates to a new state $y$ drawn from $K(x, \\cdot)$.\n", + "\n", + "Now we take $y$ as the new state for the process and repeat.\n", + "\n", + "Here is the same algorithm written more explicitly: \n", + "\n", + "```{proof:algorithm} Embedded Jump Chain Algorithm\n", + ":label: ejc_algo\n", + "\n", + "**Inputs** $\\psi \\in \\dD$, rate function $\\lambda$, Markov matrix $K$\n", + "\n", + "**Outputs** Markov chain $(X_t)$\n", + "\n", + "1. draw $Y_0$ from $\\psi$, set $J_0 = 0$ and $k=1$.\n", + "1. Draw $W_k$ independently from Exp$(\\lambda(Y_{k-1}))$.\n", + "1. Set $J_k = J_{k-1} + W_k$.\n", + "1. Set $X_t = Y_{k-1}$ for $t$ in $[J_{k-1}, J_k)$.\n", + "1. Draw $Y_k$ from $K(Y_{k-1}, \\cdot)$.\n", + "1. Set $k = k+1$ and go to step 2.\n", + "\n", + "```\n", + "\n", + "The sequence $(W_k)$ is drawn as an IID sequence and $(W_k)$ and $(Y_k)$ are\n", + "drawn independently.\n", + "\n", + "The restriction $K(x,x) = 0$ for all $x$ implies that $(X_t)$ actually jumps at each jump time.\n", + "\n", + "\n", + "## Computing the Semigroup\n", + "\n", + "For the jump process $(X_t)$ with time varying intensities described in the\n", + "jump chain algorithm, calculating the Markov semigroup is not a trivial exercise.\n", + "\n", + "The approach we adopt is\n", + "\n", + "1. use probabilistic reasoning to obtain an integral equation that the\n", + " semigroup must satisfy.\n", + "1. Convert the integral equation into a differential equation that is easier\n", + " to work with.\n", + "1. Solve this differential equation to obtain the Markov semigroup $(P_t)$.\n", + "\n", + "The differential equation in question has a special name: the Kolmogorov backward equation.\n", + "\n", + "\n", + "### An Integral Equation\n", + "\n", + "\n", + "Here is the first step in the sequence listed above.\n", + "\n", + "\n", + "```{proof:lemma} An Integral Equation\n", + "\n", + "The semigroup $(P_t)$ of the jump chain with rate function $\\lambda$ and Markov matrix $K$ obeys the integral equation \n", + "\n", + "$$\n", + " P_t(x, y) = e^{-t \\lambda(x)} I(x, y)\n", + " + \\lambda(x) \n", + " \\int_0^t (K P_{t-\\tau})(x, y) e^{- \\tau \\lambda(x)} d \\tau\n", + "$$ (kbinteg)\n", + "\n", + "for all $t \\geq 0$ and $x, y$ in $S$.\n", + "```\n", + "\n", + "Here $(P_t)$ is the Markov semigroup of $(X_t)$, the process constructed via\n", + "{proof:ref}`ejc_algo`, while $K P_{t-\\tau}$ is the matrix product of $K$ and\n", + "$P_{t-\\tau}$.\n", + "\n", + "```{proof:proof}\n", + "\n", + "Conditioning implicitly on $X_0 = x$, the semigroup $(P_t)$ must satisfy \n", + "\n", + "$$\n", + " P_t(x, y) \n", + " = \\PP\\{X_t = y\\}\n", + " = \\PP\\{X_t = y, \\; J_1 > t \\}\n", + " + \\PP\\{X_t = y, \\; J_1 \\leq t \\}\n", + "$$ (pt_split)\n", + "\n", + "Regarding the first term on the right hand side of {eq}`pt_split`, we have \n", + "\n", + "$$\n", + " \\PP\\{X_t = y, \\; J_1 > t \\}\n", + " = I(x, y) P\\{J_1 > t \\}\n", + " = I(x, y) e^{- t \\lambda(x)}\n", + "$$ (pt_first)\n", + "\n", + "where $I(x, y) = \\mathbb 1\\{x = y\\}$.\n", + "\n", + "For the second term on the right hand side of {eq}`pt_split`, we have \n", + "\n", + "$$\n", + " \\PP\\{X_t = y, \\; J_1 > t \\}\n", + " = \\EE \n", + " \\left[\n", + " \\mathbb 1\\{J_1 > t\\} \\PP\\{X_t = y \\,|\\, W_1, Y_1\\}\n", + " \\right]\n", + " = \\EE \n", + " \\left[\n", + " \\mathbb 1\\{J_1 > t\\} P_{t - J_1} (Y_1, y) \n", + " \\right]\n", + "$$\n", + "\n", + "Evaluating the expectation and using the independence of $J_1$ and $Y_1$, this becomes\n", + "\n", + "$$\n", + "\\begin{aligned}\n", + " \\PP\\{X_t = y, \\; J_1 > t \\}\n", + " & = \\int_0^\\infty\n", + " \\mathbb 1\\{\\tau > t\\}\n", + " \\sum_z K(x, z) P_{t - \\tau} (z, y) \\lambda(x) e^{-\\tau \\lambda(x)} \n", + " d \\tau\n", + " \\\\\n", + " & = \\lambda(x)\n", + " \\int_0^t\n", + " \\sum_z K(x, z) P_{t - \\tau} (z, y) e^{-\\tau \\lambda(x)} \n", + " d \\tau\n", + "\\end{aligned}\n", + "$$\n", + "\n", + "Combining this result with {eq}`pt_split` and {eq}`pt_first` gives\n", + "{eq}`kbinteg`.\n", + "```\n", + "\n", + "\n", + "### Kolmogorov's Differential Equation\n", + "\n", + "We have now confirmed that the semigroup $(P_t)$ associated with the jump\n", + "chain process $(X_t)$ satisfies {eq}`kbinteg`.\n", + "\n", + "Equation {eq}`kbinteg` is important but we can simplify it further without\n", + "losing information by taking the time derivative.\n", + "\n", + "This leads to our main result for the lecture\n", + "\n", + "\n", + "```{proof:theorem} Kolmogorov Backward Equation\n", + "\n", + "The semigroup $(P_t)$ of the jump chain with rate function $\\lambda$ and Markov matrix $K$ satisfies the **Kolmogorov backward equation**\n", + "\n", + "$$\n", + " P'_t = Q P_t \n", + " \\quad \\text{where } \\;\n", + " Q(x, y) := \\lambda(x) (K(x, y) - I(x, y))\n", + "$$ (kolbackeq)\n", + "```\n", + "\n", + "The derivative on the left hand side of {eq}`kolbackeq` is taken element by\n", + "element, with respect to $t$, so that\n", + "\n", + "$$\n", + " P'_t(x, y) = \\left( \\frac{d}{dt} P_t(x, y) \\right)\n", + " \\qquad ((x, y) \\in S \\times S)\n", + "$$\n", + "\n", + "The proof that differentiating {eq}`kbinteg` yields {eq}`kolbackeq` is an\n", + "important exercise (see below).\n", + "\n", + "\n", + "\n", + "### Exponential Solution\n", + "\n", + "The Kolmogorov backward equation is a matrix-valued differential equation.\n", + "\n", + "Recall that, for a scalar differential equation $y'_t = a y_t$ with constant\n", + "$a$ and initial condition $y_0$, the solution is $y_t = e^{ta} y_0$.\n", + "\n", + "This, along with $P_0 = I$, encourages us to guess that the solution to\n", + "Kolmogorov's backward equation {eq}`kolbackeq` is\n", + "\n", + "$$\n", + " P_t = e^{t Q} \n", + "$$ (expsol)\n", + "\n", + "where the right hand side is the [matrix exponential](https://en.wikipedia.org/wiki/Matrix_exponential), with definition \n", + "\n", + "$$\n", + " e^{tQ} \n", + " = \\sum_{k \\geq 0} \\frac{1}{k!} (tQ)^k\n", + " = I + tQ + \\frac{t^2}{2!} Q^2 + \\cdots\n", + "$$ (expofun)\n", + "\n", + "Working element by element, it is not difficult to confirm that \n", + "the derivative of the exponential function $t \\mapsto e^{tQ}$ is\n", + "\n", + "$$\n", + " \\frac{d}{dt} e^{t Q} = Q e^{t Q} = e^{t Q} Q\n", + "$$ (expoderiv)\n", + "\n", + "Hence, differentiating {eq}`expsol` gives $P'_t = Q e^{t Q} = Q P_t$, which\n", + "convinces us that the exponential solution satisfies {eq}`kolbackeq`. \n", + "\n", + "Notice that our solution\n", + "\n", + "$$\n", + " P_t = e^{t Q} \n", + " \\quad \\text{where } \\;\n", + " Q(x, y) := \\lambda(x) (K(x, y) - I(x, y))\n", + "$$ (psolq)\n", + "\n", + "for the semigroup of the jump process $(X_t)$ associated with the jump matrix\n", + "$K$ and the jump intensity function $\\lambda \\colon S \\to (0, \\infty)$ is\n", + "consistent with our earlier result.\n", + "\n", + "In particular, we {ref}`showed ` that, for the model with\n", + "constant jump intensity $\\lambda$, we have $P_t = e^{t \\lambda (K - I)}$.\n", + "\n", + "This is obviously a special case of {eq}`psolq`.\n", + "\n", + "\n", + "\n", + "## Properties of the Solution\n", + "\n", + "Let's investigate further the properties of the exponential solution.\n", + "\n", + "### Checking the Transition Semigroup Properties\n", + "\n", + "While we have confirmed that $P_t = e^{t Q}$ solves the Kolmogorov backward\n", + "equation, we still need to check that this solution is a Markov semigroup.\n", + "\n", + "```{proof:lemma} From Jump Chain to Semigroup\n", + ":label: jctosg\n", + "\n", + "Let $\\lambda$ map $S$ to $\\RR_+$ and let $K$ be a Markov matrix on $S$.\n", + "If $P_t = e^{t Q}$ for all $t \\geq 0$, where\n", + "$Q(x, y) = \\lambda(x) (K(x, y) - I(x, y))$, then $(P_t)$ is a Markov\n", + "semigroup on $S$.\n", + "```\n", + "\n", + "\n", + "```{proof:proof}\n", + "Observe first that $Q$ has zero row sums, since\n", + "\n", + "$$\n", + " \\sum_y Q(x, y) \n", + " = \\lambda(x) \\sum_y (K(x, y) - I(x, y))\n", + " = 0\n", + "$$\n", + "\n", + "As a small exercise, you can check that the following is true\n", + "\n", + "$$\n", + " Q \\text{ has zero row sums }\n", + " \\iff\n", + " Q^k 1 = 0 \\text{ for all } k \\geq 1\n", + "$$ (zrsnec)\n", + "\n", + "This implies that $Q^k 1 = 0$ for all $k$ and, as a result, for any $t \\geq\n", + "0$,\n", + "\n", + "$$\n", + " P_t 1\n", + " = e^{tQ} 1\n", + " = I1 + tQ1 + \\frac{t^2}{2!} Q^2 1 + \\cdots\n", + " = I1 = 1\n", + "$$\n", + "\n", + "In other words, each $P_t$ has unit row sums. \n", + "\n", + "Next we check positivity of all elements of $P_t$ (which can easily fail for\n", + "matrix exponentials).\n", + "\n", + "To this end, adopting an argument from {cite}`stroock2013introduction`, we\n", + "set $m := \\max_x \\lambda(x)$ and $\\hat P := I + Q / m$.\n", + "\n", + "It is not difficult to check that $\\hat P$ is a Markov matrix and $Q = m( \\hat\n", + "P - I)$.\n", + "\n", + "Recalling that, for matrix exponentials, $e^{A+B} = e^A e^B$ whenever $AB =\n", + "BA$, we have\n", + "\n", + "$$\n", + " e^{tQ} \n", + " = e^{tm (\\hat P - I)} \n", + " = e^{-tm I} e^{tm \\hat P}\n", + " = e^{-tm} \n", + " \\left( \n", + " I + tm \\hat P + \\frac{(tm)^2}{2!} \\hat P^2 + \\cdots\n", + " \\right)\n", + "$$\n", + "\n", + "It is clear from this representation that all entries of $e^{tQ}$ are\n", + "nonnegative.\n", + "\n", + "Finally, we need to check the continuity condition $P_t(x, y) \\to I(x,y)$ as\n", + "$t \\to 0$, which is also part of the definition of a Markov semigroup.\n", + "This is immediate, in the present case, because the exponential function is\n", + "continuous, and hence $P_t = e^{tQ} \\to e^0 = I$.\n", + "```\n", + "\n", + "We can now be reassured that our solution to the Kolmogorov backward equation\n", + "is indeed a Markov semigroup.\n", + "\n", + "\n", + "### Uniqueness\n", + "\n", + "Might there be another, entirely different Markov semigroup that also\n", + "satisfies the Kolmogorov backward equation?\n", + "\n", + "The answer is no: linear ODEs in finite dimensional vector space with\n", + "constant coefficients and fixed initial conditions (in this case $P_0 = I$)\n", + "have unique solutions.\n", + "\n", + "In fact it's not hard to supply a proof --- see the exercises.\n", + "\n", + "\n", + "\n", + "## Application: The Inventory Model\n", + "\n", + "Let us look at a modified version of the inventory model where jump\n", + "intensities depend on the state.\n", + "\n", + "In particular, the wait time for new inventory will now be exponential at rate\n", + "$\\gamma$.\n", + "\n", + "The arrival rate for customers will still be denoted by $\\lambda$ and allowed\n", + "to differ from $\\gamma$.\n", + "\n", + "For parameters we take" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "α = 0.6\n", + "λ = 0.5\n", + "γ = 0.1\n", + "b = 10" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Our plan is to investigate the distribution $\\psi_T$ of $X_T$ at $T=30$.\n", + "\n", + "We will do this by simulating many independent draws of $X_T$ and\n", + "histogramming them.\n", + "\n", + "(In the exercises you are asked to calculate $\\psi_T$ a different way, via\n", + "{eq}`psolq`.)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "@njit\n", + "def draw_X(T, X_0, max_iter=5000):\n", + " \"\"\"\n", + " Generate one draw of X_T given X_0.\n", + " \"\"\"\n", + "\n", + " J, Y = 0, X_0\n", + " m = 0\n", + "\n", + " while m < max_iter:\n", + " s = 1/γ if Y == 0 else 1/λ\n", + " W = np.random.exponential(scale=s) # W ~ E(λ)\n", + " J += W\n", + " if J >= T:\n", + " return Y\n", + " # Otherwise update Y\n", + " if Y == 0:\n", + " Y = b\n", + " else:\n", + " U = np.random.geometric(α)\n", + " Y = Y - min(Y, U)\n", + " m += 1\n", + "\n", + "\n", + "@njit\n", + "def independent_draws(T=10, num_draws=100):\n", + " \"Generate a vector of independent draws of X_T.\"\n", + "\n", + " draws = np.empty(num_draws, dtype=np.int64)\n", + "\n", + " for i in range(num_draws):\n", + " X_0 = np.random.binomial(b+1, 0.25)\n", + " draws[i] = draw_X(T, X_0)\n", + "\n", + " return draws" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEGCAYAAAB1iW6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAOlElEQVR4nO3dfZBdB1nH8e+PxI4CMqiJb0khQQo1I0U6sRQBrbw4LagBxKEVy7udOpQ3ZaQ4jo7DH8DoIDJWMpkaXqRSClQNNVKZCoNYwGyhtrRQzRSkS4FuQeVFh5Ly+Mc9heu6yd4k9+7tPv1+Zjp7z7lnz31Os/PNydm9Z1NVSJLWv3vNewBJ0nQYdElqwqBLUhMGXZKaMOiS1MTGeb3wpk2batu2bfN6eUlal6655prbq2rzSs/NLejbtm1jYWFhXi8vSetSkn8/3HNecpGkJgy6JDVh0CWpCYMuSU0YdElqwqBLUhMGXZKaMOiS1IRBl6Qm5vZO0ePxysuvn9m+X/20h81s35I0S56hS1ITBl2SmjDoktSEQZekJgy6JDVh0CWpCYMuSU0YdElqwqBLUhMGXZKaMOiS1IRBl6QmDLokNWHQJakJgy5JTUwU9CRnJrkpycEkFx5hu59KcmeSp09vREnSJFYNepINwEXAWcAO4JwkOw6z3WuBK6c9pCRpdZOcoZ8GHKyqm6vqDuBSYNcK270IeDdw2xTnkyRNaJKgbwFuGVteHNZ9W5ItwFOB3dMbTZJ0NCYJelZYV8uWXw+8oqruPOKOkvOSLCRZWFpamnRGSdIEJvkl0YvAiWPLW4Fbl22zE7g0CcAm4ElJDlXVX49vVFV7gD0AO3fuXP6XgiTpOEwS9APASUm2A58DzgZ+dXyDqtp+1+MkbwauWB5zSdJsrRr0qjqU5AJGP72yAdhbVTckOX943uvmknQ3MMkZOlW1H9i/bN2KIa+q5xz/WJKko+U7RSWpCYMuSU0YdElqwqBLUhMGXZKaMOiS1IRBl6QmDLokNWHQJakJgy5JTRh0SWrCoEtSEwZdkpow6JLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktSEQZekJgy6JDVh0CWpCYMuSU0YdElqwqBLUhMGXZKaMOiS1IRBl6QmDLokNWHQJakJgy5JTRh0SWrCoEtSEwZdkpow6JLUxERBT3JmkpuSHExy4QrP70pyXZJrkywkecz0R5UkHcnG1TZIsgG4CHgisAgcSLKvqm4c2+wqYF9VVZJTgMuAk2cxsCRpZZOcoZ8GHKyqm6vqDuBSYNf4BlX1taqqYfE+QCFJWlOTBH0LcMvY8uKw7v9I8tQknwL+FnjeSjtKct5wSWZhaWnpWOaVJB3GJEHPCuv+3xl4Vf1VVZ0MPAV41Uo7qqo9VbWzqnZu3rz56CaVJB3RJEFfBE4cW94K3Hq4javqg8CPJdl0nLNJko7CJEE/AJyUZHuSE4CzgX3jGyR5cJIMj08FTgC+NO1hJUmHt+pPuVTVoSQXAFcCG4C9VXVDkvOH53cDvww8K8k3gf8BnjH2TVJJ0hpYNegAVbUf2L9s3e6xx68FXjvd0SRJR8N3ikpSEwZdkpow6JLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktSEQZekJgy6JDVh0CWpCYMuSU0YdElqwqBLUhMGXZKaMOiS1IRBl6QmDLokNWHQJakJgy5JTRh0SWrCoEtSEwZdkpow6JLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktSEQZekJgy6JDVh0CWpiYmCnuTMJDclOZjkwhWef2aS64b/rk7y8OmPKkk6klWDnmQDcBFwFrADOCfJjmWbfRr42ao6BXgVsGfag0qSjmySM/TTgINVdXNV3QFcCuwa36Cqrq6q/xgWPwJsne6YkqTVTBL0LcAtY8uLw7rDeT7wdys9keS8JAtJFpaWliafUpK0qkmCnhXW1YobJj/HKOivWOn5qtpTVTuraufmzZsnn1KStKqNE2yzCJw4trwVuHX5RklOAS4GzqqqL01nPEnSpCY5Qz8AnJRke5ITgLOBfeMbJHkAcDlwblX96/THlCStZtUz9Ko6lOQC4EpgA7C3qm5Icv7w/G7g94AfAP4sCcChqto5u7ElSctNcsmFqtoP7F+2bvfY4xcAL5juaJKko+E7RSWpCYMuSU0YdElqwqBLUhMGXZKaMOiS1IRBl6QmDLokNWHQJakJgy5JTRh0SWrCoEtSEwZdkpow6JLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktSEQZekJgy6JDVh0CWpCYMuSU0YdElqwqBLUhMGXZKaMOiS1IRBl6QmDLokNWHQJakJgy5JTRh0SWrCoEtSEwZdkpow6JLUxERBT3JmkpuSHExy4QrPn5zkw0m+keTl0x9TkrSajattkGQDcBHwRGAROJBkX1XdOLbZl4EXA0+ZyZSSpFVNcoZ+GnCwqm6uqjuAS4Fd4xtU1W1VdQD45gxmlCRNYJKgbwFuGVteHNYdtSTnJVlIsrC0tHQsu5AkHcYkQc8K6+pYXqyq9lTVzqrauXnz5mPZhSTpMCYJ+iJw4tjyVuDW2YwjSTpWkwT9AHBSku1JTgDOBvbNdixJ0tFa9adcqupQkguAK4ENwN6quiHJ+cPzu5P8MLAA3A/4VpKXAjuq6isznF2SNGbVoANU1X5g/7J1u8cef4HRpRhJ0pxMFHRJuqd55eXXz2zfr37aw2ayX9/6L0lNGHRJasKgS1ITBl2SmjDoktSEQZekJgy6JDVh0CWpCYMuSU0YdElqwqBLUhMGXZKaMOiS1IRBl6QmDLokNeH90Cc0q3sjH+6+yF1e73Cv2f315mGtv2bW2j3hz/B4GXRpRgzssbm7HN965CUXSWrCoEtSEwZdkpow6JLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktSEQZekJgy6JDVh0CWpCYMuSU0YdElqwqBLUhMGXZKaMOiS1MREQU9yZpKbkhxMcuEKzyfJG4bnr0ty6vRHlSQdyapBT7IBuAg4C9gBnJNkx7LNzgJOGv47D3jjlOeUJK1ikjP004CDVXVzVd0BXArsWrbNLuCtNfIR4P5JfmTKs0qSjiBVdeQNkqcDZ1bVC4blc4FHVtUFY9tcAbymqj40LF8FvKKqFpbt6zxGZ/AADwVumtaBrGITcPsavdY8eHzrX/dj7H58sHbH+MCq2rzSExsn+OSssG753wKTbENV7QH2TPCaU5Vkoap2rvXrrhWPb/3rfozdjw/uHsc4ySWXReDEseWtwK3HsI0kaYYmCfoB4KQk25OcAJwN7Fu2zT7gWcNPu5wO/FdVfX7Ks0qSjmDVSy5VdSjJBcCVwAZgb1XdkOT84fndwH7gScBB4L+B585u5GOy5pd51pjHt/51P8buxwd3g2Nc9ZuikqT1wXeKSlITBl2Smmgd9NVuWbDeJTkxyfuTfDLJDUleMu+ZZiHJhiQfH97v0EqS+yd5V5JPDX+Oj5r3TNOW5GXD1+cnkrw9yXfPe6bjkWRvktuSfGJs3fcneV+Sfxs+ft88Zmsb9AlvWbDeHQJ+q6p+HDgdeGHDYwR4CfDJeQ8xI38CvLeqTgYeTrPjTLIFeDGws6p+gtEPVpw936mO25uBM5etuxC4qqpOAq4altdc26Az2S0L1rWq+nxVfWx4/FVGMdgy36mmK8lW4MnAxfOeZdqS3A/4GeDPAarqjqr6z/lONRMbge9JshG4N+v8PSpV9UHgy8tW7wLeMjx+C/CUNR1q0DnoW4BbxpYXaRa7cUm2AY8APjrfSabu9cBvA9+a9yAz8CBgCXjTcEnp4iT3mfdQ01RVnwP+CPgs8HlG71H5+/lONRM/dNd7b4aPPziPIToHfaLbEXSQ5L7Au4GXVtVX5j3PtCT5BeC2qrpm3rPMyEbgVOCNVfUI4OvM6Z/qszJcS94FbAd+FLhPkl+b71R9dQ76PeJ2BEm+i1HML6mqy+c9z5Q9GvilJJ9hdMnscUneNt+RpmoRWKyqu/5V9S5Gge/kCcCnq2qpqr4JXA789JxnmoUv3nWH2eHjbfMYonPQJ7llwbqWJIyuv36yql4373mmrapeWVVbq2oboz+/f6iqNmd3VfUF4JYkDx1WPR64cY4jzcJngdOT3Hv4en08zb7xO9gHPHt4/Gzgb+YxxCR3W1yXDnfLgjmPNW2PBs4Frk9y7bDud6pq/xxn0tF5EXDJcNJxM3e/22Ycl6r6aJJ3AR9j9FNZH+du8Bb545Hk7cAZwKYki8DvA68BLkvyfEZ/if3KXGbzrf+S1EPnSy6SdI9i0CWpCYMuSU0YdElqwqBLUhMGXetWkqvX8LXOSNLxDTFqxKBr3aqqtQzsGRzlOxyHm1FJa8aga91K8rXh4xlJPjB2X/FLhl9YflaSy8a2PyPJe4bHP5/kw0k+luSdw/1wSPKZJH8wrL8+ycnDjc/OB16W5Nokj03ywCRXJblu+PiA4fPfnOR1Sd4P/OFwf+zNw3P3Gu7Nv2lN/0fpHsOgq4tHAC9ldO/7BzF6F+37GL3t/K47GD4DeMcQ1N8FnlBVpwILwG+O7ev2Yf0bgZdX1WeA3cAfV9VPVtU/An8KvLWqTgEuAd4w9vkPGfb9MuBtwDOH9U8A/qWqbp/uoUsjBl1d/HNVLVbVt4BrgW1VdQh4L/CLw+WPJzO6x8bpjML/T8MtE54NPHBsX3fd5OwaYNthXu9RwF8Oj/8CeMzYc++sqjuHx3uBZw2Pnwe86dgOT1qd1/jUxTfGHt/Jd7623wG8kNEvJDhQVV8dbhL1vqo6Z5V9je9nNeP30Pj6t1dW3ZLki0keBzyS75ytS1PnGbq6+wCjW9L+OqO4A3wEeHSSBwMMdwJ8yCr7+SrwvWPLV/OdX6X2TOBDR/jcixlderls7MxdmjqDrtaGgF7B6HfLXjGsWwKeA7w9yXWMAn/yKrt6D/DUu74pyuj3ZD53+PxzGf3e08PZB9wXL7doxrzbojRjSXYy+obqY+c9i3rzGro0Q0kuBH4Dr51rDXiGLklNeA1dkpow6JLUhEGXpCYMuiQ1YdAlqYn/Bc7Qn0BUogrwAAAAAElFTkSuQmCC\n", + "text/plain": [ + "

" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/kolmogorov_bwd_6_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "T = 30\n", + "n = b + 1 \n", + "draws = independent_draws(T, num_draws=100_000)\n", + "fig, ax = plt.subplots()\n", + "\n", + "ax.bar(range(n), [np.mean(draws == i) for i in range(n)], width=0.8, alpha=0.6)\n", + "ax.set(xlabel=\"inventory\")\n", + "\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you experiment with the code above, you will see that the large amount of\n", + "mass on zero is due to the low arrival rate $\\gamma$ for inventory.\n", + "\n", + "\n", + "\n", + "\n", + "## Exercises\n", + "\n", + "### Exercise 1\n", + "\n", + "In the discussion above, we generated an approximation of $\\psi_T$ when\n", + "$T=30$, the initial condition is Binomial$(n, 0.25)$ and parameters\n", + "are set to" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "α = 0.6\n", + "λ = 0.5\n", + "γ = 0.1\n", + "b = 10" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The calculation was done by simulating independent draws and histogramming.\n", + "\n", + "Try to generate the same figure using {eq}`psolq` instead, modifying code from\n", + "{ref}`our lecture ` on the Markov property.\n", + "\n", + "### Exercise 2\n", + "\n", + "Prove that differentiating {eq}`kbinteg` at each $(x, y)$ yields {eq}`kolbackeq`.\n", + "\n", + "### Exercise 3\n", + "\n", + "We claimed above that the solution $P_t = e^{t Q}$ is the unique\n", + "Markov semigroup satisfying the backward equation $P'_t = Q P_t$.\n", + "\n", + "Try to supply a proof.\n", + "\n", + "(This is not an easy exercise but worth thinking about in any case.)\n", + "\n", + "\n", + "## Solutions\n", + "\n", + "### Solution to Exercise 1\n", + "\n", + "Here is one solution:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEGCAYAAAB1iW6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAOiklEQVR4nO3dfZBdB1nH8e+PxI4CMqhZ35KWBCnUjhTpxFIEtPLitKAGEIdWLO926lDelJHiODoOfwCjg8hYyWRqeZFKKVA11EhlKgxiAZNCbSlQzRSkS4GmoPKiQ0l5/OOewnXdl5vk3r3dp9/PTGbvOffsuc9JMt+cnN17NlWFJGnju9e8B5AkTYdBl6QmDLokNWHQJakJgy5JTWye1wtv2bKltm/fPq+Xl6QN6dprr729qhaWe25uQd++fTsHDhyY18tL0oaU5N9Xes5LLpLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktTE3N4peixeccUNM9v3q5760JntW5JmyTN0SWrCoEtSEwZdkpow6JLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktSEQZekJgy6JDUxUdCTnJnkpiQHk1y4ynY/leTOJE+b3oiSpEmsGfQkm4CLgLOAk4Fzkpy8wnavAa6a9pCSpLVNcoZ+GnCwqm6uqjuAy4Bdy2z3QuBdwG1TnE+SNKFJgr4VuGVseXFY921JtgJPAXavtqMk5yU5kOTAoUOHjnRWSdIqJgl6lllXS5ZfB7y8qu5cbUdVtaeqdlbVzoWFhUlnlCRNYJKfKboIHD+2vA24dck2O4HLkgBsAZ6Y5HBV/fVUppQkrWmSoO8HTkyyA/gccDbwq+MbVNWOux4neRNwpTGXpPW1ZtCr6nCSCxh998om4JKqujHJ+cPzq143lyStj0nO0KmqfcC+JeuWDXlVPfvYx5IkHSnfKSpJTRh0SWrCoEtSEwZdkpow6JLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktSEQZekJgy6JDVh0CWpCYMuSU0YdElqwqBLUhMGXZKaMOiS1IRBl6QmDLokNWHQJakJgy5JTRh0SWrCoEtSEwZdkpow6JLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktSEQZekJiYKepIzk9yU5GCSC5d5fleS65Ncl+RAkkdPf1RJ0mo2r7VBkk3ARcATgEVgf5K9VfWJsc2uBvZWVSU5BbgcOGkWA0uSljfJGfppwMGqurmq7gAuA3aNb1BVX6uqGhbvAxSSpHU1SdC3AreMLS8O6/6PJE9J8ingb4HnLrejJOcNl2QOHDp06GjmlSStYJKgZ5l1/+8MvKr+qqpOAp4MvHK5HVXVnqraWVU7FxYWjmxSSdKqJgn6InD82PI24NaVNq6qDwA/lmTLMc4mSToCkwR9P3Bikh1JjgPOBvaOb5DkQUkyPD4VOA740rSHlSStbM3vcqmqw0kuAK4CNgGXVNWNSc4fnt8N/DLwzCTfBP4HePrYF0klSetgzaADVNU+YN+SdbvHHr8GeM10R5MkHQnfKSpJTRh0SWrCoEtSEwZdkpow6JLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktSEQZekJgy6JDVh0CWpCYMuSU0YdElqwqBLUhMGXZKaMOiS1IRBl6QmDLokNWHQJakJgy5JTRh0SWrCoEtSEwZdkpow6JLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktSEQZekJiYKepIzk9yU5GCSC5d5/hlJrh9+XZPkYdMfVZK0mjWDnmQTcBFwFnAycE6Sk5ds9mngZ6vqFOCVwJ5pDypJWt0kZ+inAQer6uaqugO4DNg1vkFVXVNV/zEsfhjYNt0xJUlrmSToW4FbxpYXh3UreR7wd8cylCTpyG2eYJsss66W3TD5OUZBf/QKz58HnAdwwgknTDiiJGkSk5yhLwLHjy1vA25dulGSU4CLgV1V9aXldlRVe6pqZ1XtXFhYOJp5JUkrmCTo+4ETk+xIchxwNrB3fIMkJwBXAOdW1b9Of0xJ0lrWvORSVYeTXABcBWwCLqmqG5OcPzy/G/g94AeAP0sCcLiqds5ubEnSUpNcQ6eq9gH7lqzbPfb4+cDzpzuaJOlI+E5RSWrCoEtSEwZdkpow6JLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktSEQZekJgy6JDVh0CWpCYMuSU0YdElqwqBLUhMGXZKaMOiS1IRBl6QmDLokNWHQJakJgy5JTRh0SWrCoEtSEwZdkpow6JLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktSEQZekJgy6JDVh0CWpiYmCnuTMJDclOZjkwmWePynJh5J8I8nLpj+mJGktm9faIMkm4CLgCcAisD/J3qr6xNhmXwZeBDx5JlNKktY0yRn6acDBqrq5qu4ALgN2jW9QVbdV1X7gmzOYUZI0gUmCvhW4ZWx5cVh3xJKcl+RAkgOHDh06ml1IklYwSdCzzLo6mherqj1VtbOqdi4sLBzNLiRJK5gk6IvA8WPL24BbZzOOJOloTRL0/cCJSXYkOQ44G9g727EkSUdqze9yqarDSS4ArgI2AZdU1Y1Jzh+e353kh4EDwP2AbyV5CXByVX1lhrNLksasGXSAqtoH7FuybvfY4y8wuhQjSZqTiYIuSfc0r7jihpnt+1VPfehM9utb/yWpCYMuSU0YdElqwqBLUhMGXZKaMOiS1IRBl6QmDLokNWHQJakJgy5JTRh0SWrCoEtSEwZdkpow6JLUhEGXpCa8H/qEZnVv5JXui9zl9VZ6ze6vN8vX9PXW7zU3Gs/QJakJgy5JTRh0SWrCoEtSEwZdkpow6JLUhEGXpCYMuiQ1YdAlqQmDLklNGHRJasKgS1ITBl2SmjDoktSEQZekJgy6JDVh0CWpCYMuSU0YdElqwqBLUhMGXZKamCjoSc5MclOSg0kuXOb5JHn98Pz1SU6d/qiSpNWsGfQkm4CLgLOAk4Fzkpy8ZLOzgBOHX+cBb5jynJKkNUxyhn4acLCqbq6qO4DLgF1LttkFvKVGPgzcP8mPTHlWSdIqUlWrb5A8DTizqp4/LJ8LPKKqLhjb5krg1VX1wWH5auDlVXVgyb7OY3QGD/AQ4KZpHcgatgC3r9NrzYPHt/F1P8buxwfrd4wPqKqF5Z7YPMEnZ5l1S/8VmGQbqmoPsGeC15yqJAeqaud6v+568fg2vu7H2P344O5xjJNcclkEjh9b3gbcehTbSJJmaJKg7wdOTLIjyXHA2cDeJdvsBZ45fLfL6cB/VdXnpzyrJGkVa15yqarDSS4ArgI2AZdU1Y1Jzh+e3w3sA54IHAT+G3jO7EY+Kut+mWedeXwbX/dj7H58cDc4xjW/KCpJ2hh8p6gkNWHQJamJ1kFf65YFG12S45O8L8knk9yY5MXznmkWkmxK8rHh/Q6tJLl/kncm+dTw5/jIec80bUleOvz9/HiStyX57nnPdCySXJLktiQfH1v3/Unem+Tfho/fN4/Z2gZ9wlsWbHSHgd+qqh8HTgde0PAYAV4MfHLeQ8zInwDvqaqTgIfR7DiTbAVeBOysqp9g9I0VZ893qmP2JuDMJesuBK6uqhOBq4flddc26Ex2y4INrao+X1UfHR5/lVEMts53qulKsg14EnDxvGeZtiT3A34G+HOAqrqjqv5zvlPNxGbge5JsBu7NBn+PSlV9APjyktW7gDcPj98MPHldhxp0DvpW4Jax5UWaxW5cku3Aw4GPzHeSqXsd8NvAt+Y9yAw8EDgEvHG4pHRxkvvMe6hpqqrPAX8EfBb4PKP3qPz9fKeaiR+66703w8cfnMcQnYM+0e0IOkhyX+BdwEuq6ivznmdakvwCcFtVXTvvWWZkM3Aq8Iaqejjwdeb0X/VZGa4l7wJ2AD8K3CfJr813qr46B/0ecTuCJN/FKOaXVtUV855nyh4F/FKSzzC6ZPbYJG+d70hTtQgsVtVd/6t6J6PAd/J44NNVdaiqvglcAfz0nGeahS/edYfZ4eNt8xiic9AnuWXBhpYkjK6/frKqXjvveaatql5RVduqajujP79/qKo2Z3dV9QXgliQPGVY9DvjEHEeahc8Cpye59/D39XE0+8LvYC/wrOHxs4C/mccQk9xtcUNa6ZYFcx5r2h4FnAvckOS6Yd3vVNW+Oc6kI/NC4NLhpONm7n63zTgmVfWRJO8EPsrou7I+xt3gLfLHIsnbgDOALUkWgd8HXg1cnuR5jP4R+5W5zOZb/yWph86XXCTpHsWgS1ITBl2SmjDoktSEQZekJgy6Nqwk16zja52RpOMbYtSIQdeGVVXrGdgzOMJ3OA43o5LWjUHXhpXka8PHM5K8f+y+4pcOP7D8rCSXj21/RpJ3D49/PsmHknw0yTuG++GQ5DNJ/mBYf0OSk4Ybn50PvDTJdUkek+QBSa5Ocv3w8YTh89+U5LVJ3gf84XB/7IXhuXsN9+bfsq6/UbrHMOjq4uHASxjd+/6BjN5F+15Gbzu/6w6GTwfePgT1d4HHV9WpwAHgN8f2dfuw/g3Ay6rqM8Bu4I+r6ier6h+BPwXeUlWnAJcCrx/7/AcP+34p8FbgGcP6xwP/UlW3T/fQpRGDri7+uaoWq+pbwHXA9qo6DLwH+MXh8seTGN1j43RG4f+n4ZYJzwIeMLavu25ydi2wfYXXeyTwl8PjvwAePfbcO6rqzuHxJcAzh8fPBd54dIcnrc1rfOriG2OP7+Q7f7ffDryA0Q8k2F9VXx1uEvXeqjpnjX2N72ct4/fQ+Pq3V1bdkuSLSR4LPILvnK1LU+cZurp7P6Nb0v46o7gDfBh4VJIHAQx3AnzwGvv5KvC9Y8vX8J0fpfYM4IOrfO7FjC69XD525i5NnUFXa0NAr2T0s2WvHNYdAp4NvC3J9YwCf9Iau3o38JS7vijK6OdkPmf4/HMZ/dzTlewF7ouXWzRj3m1RmrEkOxl9QfUx855FvXkNXZqhJBcCv4HXzrUOPEOXpCa8hi5JTRh0SWrCoEtSEwZdkpow6JLUxP8Cnauf1iWBLaoAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/kolmogorov_bwd_10_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "states = np.arange(n)\n", + "I = np.identity(n)\n", + "\n", + "# Embedded jump chain matrix\n", + "K = np.zeros((n, n))\n", + "K[0, -1] = 1\n", + "for i in range(1, n):\n", + " for j in range(0, i):\n", + " if j == 0:\n", + " K[i, j] = (1 - α)**(i-1)\n", + " else:\n", + " K[i, j] = α * (1 - α)**(i-j-1)\n", + "\n", + "# Jump intensities as a function of the state\n", + "r = np.ones(n) * λ\n", + "r[0] = γ\n", + "\n", + "# Q matrix\n", + "Q = np.empty_like(K)\n", + "for i in range(n):\n", + " for j in range(n):\n", + " Q[i, j] = r[i] * (K[i, j] - I[i, j])\n", + "\n", + "def P_t(ψ, t):\n", + " return ψ @ expm(t * Q)\n", + "\n", + "ψ_0 = binom.pmf(states, n, 0.25)\n", + "ψ_T = P_t(ψ_0, T)\n", + "\n", + "fig, ax = plt.subplots()\n", + "\n", + "ax.bar(range(n), ψ_T, width=0.8, alpha=0.6)\n", + "ax.set(xlabel=\"inventory\")\n", + "\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Solution to Exercise 2\n", + "\n", + "One can easily verify that, when $f$ is a differentiable function and $\\alpha >\n", + "0$, we have\n", + "\n", + "$$\n", + " g(t) = e^{- t \\alpha} f(t)\n", + " \\quad \\implies \\quad\n", + " g'(t) = e^{- t \\alpha} f'(t) - \\alpha g(t)\n", + "$$ (gdiff)\n", + "\n", + "Note also that, with the change of variable $s = t - \\tau$, we can rewrite\n", + "{eq}`kbinteg` as\n", + "\n", + "$$\n", + " P_t(x, y) = \n", + " e^{-t \\lambda(x)} \n", + " \\left\\{ \n", + " I(x, y)\n", + " + \\lambda(x) \n", + " \\int_0^t (K P_s)(x, y) e^{s \\lambda(x)} d s\n", + " \\right\\}\n", + "$$ (kbinteg2)\n", + "\n", + "Applying {eq}`gdiff` yields\n", + "\n", + "$$\n", + " P'_t(x, y) \n", + " = e^{-t \\lambda(x)} \n", + " \\left\\{ \n", + " \\lambda(x) \n", + " (K P_t)(x, y) e^{t \\lambda(x)} \n", + " \\right\\}\n", + " - \\lambda(x) P_t(x, y)\n", + "$$\n", + "\n", + "After minor rearrangements this becomes\n", + "\n", + "$$\n", + " P'_t(x, y) \n", + " = \\lambda(x) [ (K - I) P_t](x, y) \n", + "$$\n", + "\n", + "which is identical to {eq}`kolbackeq`.\n", + "\n", + "\n", + "### Solution to Exercise 3\n", + "\n", + "Here is one proof of uniqueness.\n", + "\n", + "Suppose that $(\\hat P_t)$ is another Markov semigroup satisfying \n", + "$P'_t = Q P_t$.\n", + "\n", + "Fix $t > 0$ and let $V_s$ be defined by $V_s = P_s \\hat P_{t-s}$ for all $s\n", + "\\geq 0$.\n", + "\n", + "Note that $V_0 = \\hat P_t$ and $V_t = P_t$.\n", + "\n", + "Note also that $s \\mapsto V_s$ is differentiable, with derivative\n", + "\n", + "$$\n", + " V'_s \n", + " = P'_s \\hat P_{t-s} - P_s \\hat P'_{t-s}\n", + " = P_s Q \\hat P_{t-s} - P_s Q \\hat P_{t-s}\n", + " = 0\n", + "$$\n", + "\n", + "where, in the second last equality, we used {eq}`expoderiv`.\n", + "\n", + "\n", + "Hence $V_s$ is constant, so our previous observations $V_0 = \\hat P_t$ and $V_t = P_t$\n", + "now yield $\\hat P_t = P_t$.\n", + "\n", + "Since $t$ was arbitrary, the proof is now done." + ] + } + ], + "metadata": { + "jupytext": { + "formats": "ipynb,md:myst", + "text_representation": { + "extension": ".md", + "format_name": "myst", + "format_version": "0.9", + "jupytext_version": "1.5.0" + } + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.7" + }, + "source_map": [ + 13, + 41, + 50, + 424, + 429, + 439, + 477, + 487, + 503, + 508, + 535, + 571 + ] + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file diff --git a/_sources/kolmogorov_bwd.md b/_sources/kolmogorov_bwd.md new file mode 100644 index 0000000..c8c1fe4 --- /dev/null +++ b/_sources/kolmogorov_bwd.md @@ -0,0 +1,650 @@ +--- +jupytext: + formats: ipynb,md:myst + text_representation: + extension: .md + format_name: myst + format_version: '0.9' + jupytext_version: 1.5.0 +kernelspec: + display_name: Python 3 + language: python + name: python3 +--- + +# Kolmogorov Backward Equation + +## Overview + +As models become more complex, deriving analytical representations of the +Markov semigroup $(P_t)$ becomes harder. + +This is analogous to the idea that solutions to continuous time models often +lack analytical solutions. + +For example, when studying deterministic paths in continuous time, +infinitesimal descriptions (ODEs and PDEs) are often more intuitive and easier +to write down than the associated solutions. + +(This is one of the shining insights of mathematics, beginning with the work +of great scientists such as Isaac Newton.) + +We will see in this lecture that the same is true for continuous time Markov +chains. + +To help us focus on intuition in this lecture, rather than technicalities, the state space is assumed to be finite, with $|S|=n$. + +Later we will investigate the case where $|S| = \infty$. + +We will use the following imports + +```{code-cell} ipython3 +import numpy as np +import scipy as sp +import matplotlib.pyplot as plt +import quantecon as qe +from numba import njit + +from scipy.linalg import expm +from scipy.stats import binom +``` + +(sdji)= +## State Dependent Jump Intensities + +As we have seen, continuous time Markov chains jump between states, and hence can +have the form + +$$ + X_t = \sum_{k \geq 0} Y_k \mathbb 1\{J_k \leq t < J_{k+1}\} + \qquad (t \geq 0) +$$ + +where $(J_k)$ are jump times and $(Y_k)$ are the states at each jump. + +(We are assuming that $J_k \to \infty$ with probability one, so that $X_t$ is well +defined for all $t \geq 0$, but this is always true for when holding times are exponential and the state space is finite.) + +In the {doc}`previous lecture `, + +* the sequence $(Y_k)$ was drawn from a Markov matrix $K$ and called the embedded jump chain, while +* the holding times $W_k := J_k - J_{k-1}$ were IID and Exp$(\lambda)$ for some +constant jump intensity $\lambda$. + +In this lecture, we will generalize by allowing the jump intensity to vary +with the state. + +This difference sounds minor but in fact it will allow us to reach full generality +in our description of continuous time Markov chains, as +clarified below. + +### Motivation + +As a motivating example, recall {ref}`the inventory model `, +where we assumed that the wait time for the next customer was equal +to the wait time for new inventory. + +This assumption was made purely for convenience and seems unlikely to hold true. + +When we relax it, the jump intensities depend on the state. + +(jumpchainalgo)= +### Embedded Jump Chain Algorithm + +We start with three primitives + +1. An initial condition $\psi$, +1. a Markov matrix $K$ on $S$ satisfying $K(x, x) = 0$ for all $x \in S$ + and +1. a function $\lambda$ mapping $S$ to $(0, \infty)$. + +The process $(X_t)$ + +* starts at state $x$, draw from $\psi$, +* waits there for an exponential time $W$ with rate $\lambda(x)$ and then +* updates to a new state $y$ drawn from $K(x, \cdot)$. + +Now we take $y$ as the new state for the process and repeat. + +Here is the same algorithm written more explicitly: + +```{proof:algorithm} Embedded Jump Chain Algorithm +:label: ejc_algo + +**Inputs** $\psi \in \dD$, rate function $\lambda$, Markov matrix $K$ + +**Outputs** Markov chain $(X_t)$ + +1. draw $Y_0$ from $\psi$, set $J_0 = 0$ and $k=1$. +1. Draw $W_k$ independently from Exp$(\lambda(Y_{k-1}))$. +1. Set $J_k = J_{k-1} + W_k$. +1. Set $X_t = Y_{k-1}$ for $t$ in $[J_{k-1}, J_k)$. +1. Draw $Y_k$ from $K(Y_{k-1}, \cdot)$. +1. Set $k = k+1$ and go to step 2. + +``` + +The sequence $(W_k)$ is drawn as an IID sequence and $(W_k)$ and $(Y_k)$ are +drawn independently. + +The restriction $K(x,x) = 0$ for all $x$ implies that $(X_t)$ actually jumps at each jump time. + + +## Computing the Semigroup + +For the jump process $(X_t)$ with time varying intensities described in the +jump chain algorithm, calculating the Markov semigroup is not a trivial exercise. + +The approach we adopt is + +1. use probabilistic reasoning to obtain an integral equation that the + semigroup must satisfy. +1. Convert the integral equation into a differential equation that is easier + to work with. +1. Solve this differential equation to obtain the Markov semigroup $(P_t)$. + +The differential equation in question has a special name: the Kolmogorov backward equation. + + +### An Integral Equation + + +Here is the first step in the sequence listed above. + + +```{proof:lemma} An Integral Equation + +The semigroup $(P_t)$ of the jump chain with rate function $\lambda$ and Markov matrix $K$ obeys the integral equation + +$$ + P_t(x, y) = e^{-t \lambda(x)} I(x, y) + + \lambda(x) + \int_0^t (K P_{t-\tau})(x, y) e^{- \tau \lambda(x)} d \tau +$$ (kbinteg) + +for all $t \geq 0$ and $x, y$ in $S$. +``` + +Here $(P_t)$ is the Markov semigroup of $(X_t)$, the process constructed via +{proof:ref}`ejc_algo`, while $K P_{t-\tau}$ is the matrix product of $K$ and +$P_{t-\tau}$. + +```{proof:proof} + +Conditioning implicitly on $X_0 = x$, the semigroup $(P_t)$ must satisfy + +$$ + P_t(x, y) + = \PP\{X_t = y\} + = \PP\{X_t = y, \; J_1 > t \} + + \PP\{X_t = y, \; J_1 \leq t \} +$$ (pt_split) + +Regarding the first term on the right hand side of {eq}`pt_split`, we have + +$$ + \PP\{X_t = y, \; J_1 > t \} + = I(x, y) P\{J_1 > t \} + = I(x, y) e^{- t \lambda(x)} +$$ (pt_first) + +where $I(x, y) = \mathbb 1\{x = y\}$. + +For the second term on the right hand side of {eq}`pt_split`, we have + +$$ + \PP\{X_t = y, \; J_1 > t \} + = \EE + \left[ + \mathbb 1\{J_1 > t\} \PP\{X_t = y \,|\, W_1, Y_1\} + \right] + = \EE + \left[ + \mathbb 1\{J_1 > t\} P_{t - J_1} (Y_1, y) + \right] +$$ + +Evaluating the expectation and using the independence of $J_1$ and $Y_1$, this becomes + +$$ +\begin{aligned} + \PP\{X_t = y, \; J_1 > t \} + & = \int_0^\infty + \mathbb 1\{\tau > t\} + \sum_z K(x, z) P_{t - \tau} (z, y) \lambda(x) e^{-\tau \lambda(x)} + d \tau + \\ + & = \lambda(x) + \int_0^t + \sum_z K(x, z) P_{t - \tau} (z, y) e^{-\tau \lambda(x)} + d \tau +\end{aligned} +$$ + +Combining this result with {eq}`pt_split` and {eq}`pt_first` gives +{eq}`kbinteg`. +``` + + +### Kolmogorov's Differential Equation + +We have now confirmed that the semigroup $(P_t)$ associated with the jump +chain process $(X_t)$ satisfies {eq}`kbinteg`. + +Equation {eq}`kbinteg` is important but we can simplify it further without +losing information by taking the time derivative. + +This leads to our main result for the lecture + + +```{proof:theorem} Kolmogorov Backward Equation + +The semigroup $(P_t)$ of the jump chain with rate function $\lambda$ and Markov matrix $K$ satisfies the **Kolmogorov backward equation** + +$$ + P'_t = Q P_t + \quad \text{where } \; + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +$$ (kolbackeq) +``` + +The derivative on the left hand side of {eq}`kolbackeq` is taken element by +element, with respect to $t$, so that + +$$ + P'_t(x, y) = \left( \frac{d}{dt} P_t(x, y) \right) + \qquad ((x, y) \in S \times S) +$$ + +The proof that differentiating {eq}`kbinteg` yields {eq}`kolbackeq` is an +important exercise (see below). + + + +### Exponential Solution + +The Kolmogorov backward equation is a matrix-valued differential equation. + +Recall that, for a scalar differential equation $y'_t = a y_t$ with constant +$a$ and initial condition $y_0$, the solution is $y_t = e^{ta} y_0$. + +This, along with $P_0 = I$, encourages us to guess that the solution to +Kolmogorov's backward equation {eq}`kolbackeq` is + +$$ + P_t = e^{t Q} +$$ (expsol) + +where the right hand side is the [matrix exponential](https://en.wikipedia.org/wiki/Matrix_exponential), with definition + +$$ + e^{tQ} + = \sum_{k \geq 0} \frac{1}{k!} (tQ)^k + = I + tQ + \frac{t^2}{2!} Q^2 + \cdots +$$ (expofun) + +Working element by element, it is not difficult to confirm that +the derivative of the exponential function $t \mapsto e^{tQ}$ is + +$$ + \frac{d}{dt} e^{t Q} = Q e^{t Q} = e^{t Q} Q +$$ (expoderiv) + +Hence, differentiating {eq}`expsol` gives $P'_t = Q e^{t Q} = Q P_t$, which +convinces us that the exponential solution satisfies {eq}`kolbackeq`. + +Notice that our solution + +$$ + P_t = e^{t Q} + \quad \text{where } \; + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +$$ (psolq) + +for the semigroup of the jump process $(X_t)$ associated with the jump matrix +$K$ and the jump intensity function $\lambda \colon S \to (0, \infty)$ is +consistent with our earlier result. + +In particular, we {ref}`showed ` that, for the model with +constant jump intensity $\lambda$, we have $P_t = e^{t \lambda (K - I)}$. + +This is obviously a special case of {eq}`psolq`. + + + +## Properties of the Solution + +Let's investigate further the properties of the exponential solution. + +### Checking the Transition Semigroup Properties + +While we have confirmed that $P_t = e^{t Q}$ solves the Kolmogorov backward +equation, we still need to check that this solution is a Markov semigroup. + +```{proof:lemma} From Jump Chain to Semigroup +:label: jctosg + +Let $\lambda$ map $S$ to $\RR_+$ and let $K$ be a Markov matrix on $S$. +If $P_t = e^{t Q}$ for all $t \geq 0$, where +$Q(x, y) = \lambda(x) (K(x, y) - I(x, y))$, then $(P_t)$ is a Markov +semigroup on $S$. +``` + + +```{proof:proof} +Observe first that $Q$ has zero row sums, since + +$$ + \sum_y Q(x, y) + = \lambda(x) \sum_y (K(x, y) - I(x, y)) + = 0 +$$ + +As a small exercise, you can check that the following is true + +$$ + Q \text{ has zero row sums } + \iff + Q^k 1 = 0 \text{ for all } k \geq 1 +$$ (zrsnec) + +This implies that $Q^k 1 = 0$ for all $k$ and, as a result, for any $t \geq +0$, + +$$ + P_t 1 + = e^{tQ} 1 + = I1 + tQ1 + \frac{t^2}{2!} Q^2 1 + \cdots + = I1 = 1 +$$ + +In other words, each $P_t$ has unit row sums. + +Next we check positivity of all elements of $P_t$ (which can easily fail for +matrix exponentials). + +To this end, adopting an argument from {cite}`stroock2013introduction`, we +set $m := \max_x \lambda(x)$ and $\hat P := I + Q / m$. + +It is not difficult to check that $\hat P$ is a Markov matrix and $Q = m( \hat +P - I)$. + +Recalling that, for matrix exponentials, $e^{A+B} = e^A e^B$ whenever $AB = +BA$, we have + +$$ + e^{tQ} + = e^{tm (\hat P - I)} + = e^{-tm I} e^{tm \hat P} + = e^{-tm} + \left( + I + tm \hat P + \frac{(tm)^2}{2!} \hat P^2 + \cdots + \right) +$$ + +It is clear from this representation that all entries of $e^{tQ}$ are +nonnegative. + +Finally, we need to check the continuity condition $P_t(x, y) \to I(x,y)$ as +$t \to 0$, which is also part of the definition of a Markov semigroup. +This is immediate, in the present case, because the exponential function is +continuous, and hence $P_t = e^{tQ} \to e^0 = I$. +``` + +We can now be reassured that our solution to the Kolmogorov backward equation +is indeed a Markov semigroup. + + +### Uniqueness + +Might there be another, entirely different Markov semigroup that also +satisfies the Kolmogorov backward equation? + +The answer is no: linear ODEs in finite dimensional vector space with +constant coefficients and fixed initial conditions (in this case $P_0 = I$) +have unique solutions. + +In fact it's not hard to supply a proof --- see the exercises. + + + +## Application: The Inventory Model + +Let us look at a modified version of the inventory model where jump +intensities depend on the state. + +In particular, the wait time for new inventory will now be exponential at rate +$\gamma$. + +The arrival rate for customers will still be denoted by $\lambda$ and allowed +to differ from $\gamma$. + +For parameters we take + +```{code-cell} ipython3 +α = 0.6 +λ = 0.5 +γ = 0.1 +b = 10 +``` + +Our plan is to investigate the distribution $\psi_T$ of $X_T$ at $T=30$. + +We will do this by simulating many independent draws of $X_T$ and +histogramming them. + +(In the exercises you are asked to calculate $\psi_T$ a different way, via +{eq}`psolq`.) + +```{code-cell} ipython3 +@njit +def draw_X(T, X_0, max_iter=5000): + """ + Generate one draw of X_T given X_0. + """ + + J, Y = 0, X_0 + m = 0 + + while m < max_iter: + s = 1/γ if Y == 0 else 1/λ + W = np.random.exponential(scale=s) # W ~ E(λ) + J += W + if J >= T: + return Y + # Otherwise update Y + if Y == 0: + Y = b + else: + U = np.random.geometric(α) + Y = Y - min(Y, U) + m += 1 + + +@njit +def independent_draws(T=10, num_draws=100): + "Generate a vector of independent draws of X_T." + + draws = np.empty(num_draws, dtype=np.int64) + + for i in range(num_draws): + X_0 = np.random.binomial(b+1, 0.25) + draws[i] = draw_X(T, X_0) + + return draws +``` + +```{code-cell} ipython3 +T = 30 +n = b + 1 +draws = independent_draws(T, num_draws=100_000) +fig, ax = plt.subplots() + +ax.bar(range(n), [np.mean(draws == i) for i in range(n)], width=0.8, alpha=0.6) +ax.set(xlabel="inventory") + +plt.show() +``` + +If you experiment with the code above, you will see that the large amount of +mass on zero is due to the low arrival rate $\gamma$ for inventory. + + + + +## Exercises + +### Exercise 1 + +In the discussion above, we generated an approximation of $\psi_T$ when +$T=30$, the initial condition is Binomial$(n, 0.25)$ and parameters +are set to + +```{code-cell} ipython3 +α = 0.6 +λ = 0.5 +γ = 0.1 +b = 10 +``` + +The calculation was done by simulating independent draws and histogramming. + +Try to generate the same figure using {eq}`psolq` instead, modifying code from +{ref}`our lecture ` on the Markov property. + +### Exercise 2 + +Prove that differentiating {eq}`kbinteg` at each $(x, y)$ yields {eq}`kolbackeq`. + +### Exercise 3 + +We claimed above that the solution $P_t = e^{t Q}$ is the unique +Markov semigroup satisfying the backward equation $P'_t = Q P_t$. + +Try to supply a proof. + +(This is not an easy exercise but worth thinking about in any case.) + + +## Solutions + +### Solution to Exercise 1 + +Here is one solution: + +```{code-cell} ipython3 +states = np.arange(n) +I = np.identity(n) + +# Embedded jump chain matrix +K = np.zeros((n, n)) +K[0, -1] = 1 +for i in range(1, n): + for j in range(0, i): + if j == 0: + K[i, j] = (1 - α)**(i-1) + else: + K[i, j] = α * (1 - α)**(i-j-1) + +# Jump intensities as a function of the state +r = np.ones(n) * λ +r[0] = γ + +# Q matrix +Q = np.empty_like(K) +for i in range(n): + for j in range(n): + Q[i, j] = r[i] * (K[i, j] - I[i, j]) + +def P_t(ψ, t): + return ψ @ expm(t * Q) + +ψ_0 = binom.pmf(states, n, 0.25) +ψ_T = P_t(ψ_0, T) + +fig, ax = plt.subplots() + +ax.bar(range(n), ψ_T, width=0.8, alpha=0.6) +ax.set(xlabel="inventory") + +plt.show() +``` + + + +### Solution to Exercise 2 + +One can easily verify that, when $f$ is a differentiable function and $\alpha > +0$, we have + +$$ + g(t) = e^{- t \alpha} f(t) + \quad \implies \quad + g'(t) = e^{- t \alpha} f'(t) - \alpha g(t) +$$ (gdiff) + +Note also that, with the change of variable $s = t - \tau$, we can rewrite +{eq}`kbinteg` as + +$$ + P_t(x, y) = + e^{-t \lambda(x)} + \left\{ + I(x, y) + + \lambda(x) + \int_0^t (K P_s)(x, y) e^{s \lambda(x)} d s + \right\} +$$ (kbinteg2) + +Applying {eq}`gdiff` yields + +$$ + P'_t(x, y) + = e^{-t \lambda(x)} + \left\{ + \lambda(x) + (K P_t)(x, y) e^{t \lambda(x)} + \right\} + - \lambda(x) P_t(x, y) +$$ + +After minor rearrangements this becomes + +$$ + P'_t(x, y) + = \lambda(x) [ (K - I) P_t](x, y) +$$ + +which is identical to {eq}`kolbackeq`. + + +### Solution to Exercise 3 + +Here is one proof of uniqueness. + +Suppose that $(\hat P_t)$ is another Markov semigroup satisfying +$P'_t = Q P_t$. + +Fix $t > 0$ and let $V_s$ be defined by $V_s = P_s \hat P_{t-s}$ for all $s +\geq 0$. + +Note that $V_0 = \hat P_t$ and $V_t = P_t$. + +Note also that $s \mapsto V_s$ is differentiable, with derivative + +$$ + V'_s + = P'_s \hat P_{t-s} - P_s \hat P'_{t-s} + = P_s Q \hat P_{t-s} - P_s Q \hat P_{t-s} + = 0 +$$ + +where, in the second last equality, we used {eq}`expoderiv`. + + +Hence $V_s$ is constant, so our previous observations $V_0 = \hat P_t$ and $V_t = P_t$ +now yield $\hat P_t = P_t$. + +Since $t$ was arbitrary, the proof is now done. + + diff --git a/_sources/kolmogorov_fwd.ipynb b/_sources/kolmogorov_fwd.ipynb new file mode 100644 index 0000000..9b8e3ca --- /dev/null +++ b/_sources/kolmogorov_fwd.ipynb @@ -0,0 +1,743 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# The Kolmogorov Forward Equation\n", + "\n", + "\n", + "\n", + "## Overview\n", + "\n", + "In this lecture we approach continuous time Markov chains from a more\n", + "analytical perspective.\n", + "\n", + "The emphasis will be on describing distribution flows through vector-valued\n", + "differential equations and their solutions.\n", + "\n", + "These distribution flows show how the time $t$ distribution associated with a\n", + "given Markov chain $(X_t)$ changes over time.\n", + "\n", + "Distribution flows will be identified with initial value problems generated by autonomous linear ordinary differential equations (ODEs) in vector space.\n", + "\n", + "We will see that the solutions of these flows are described by Markov semigroups.\n", + "\n", + "This leads us back to the theory we have already constructed -- some care will\n", + "be taken to clarify all the connections.\n", + "\n", + "In order to avoid being distracted by technicalities, we continue to defer our\n", + "treatment of infinite state spaces, assuming throughout this lecture that $|S|\n", + "= n$.\n", + "\n", + "As before, $\\dD$ is the set of all distributions on $S$.\n", + "\n", + "We will use the following imports" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import scipy as sp\n", + "import matplotlib.pyplot as plt\n", + "import quantecon as qe\n", + "from numba import njit\n", + "from scipy.linalg import expm\n", + "\n", + "from matplotlib import cm\n", + "from mpl_toolkits.mplot3d import Axes3D\n", + "from mpl_toolkits.mplot3d.art3d import Poly3DCollection" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## From Difference Equations to ODEs\n", + "\n", + "{ref}`Previously ` we generated this figure, which shows how distributions evolve over time for the inventory model under a certain parameterization:\n", + "\n", + "```{glue:figure} flow_fig\n", + "Probability flows for the inventory model.\n", + "```\n", + "\n", + "(Hot colors indicate early dates and cool colors denote later dates.)\n", + "\n", + "We also learned how this flow is related to \n", + "the Kolmogorov backward equation, which is an ODE.\n", + "\n", + "In this section we examine distribution flows and their connection to \n", + "ODEs and continuous time Markov chains more systematically.\n", + "\n", + "\n", + "### Review of the Discrete Time Case\n", + "\n", + "Let $(X_t)$ be a discrete time Markov chain with Markov matrix $P$.\n", + "\n", + "{ref}`Recall that `, in the discrete time case, the distribution $\\psi_t$ of $X_t$ updates according to \n", + "\n", + "$$\n", + " \\psi_{t+1} = \\psi_t P, \n", + " \\qquad \\psi_0 \\text{ a given element of } \\dD,\n", + "$$\n", + "\n", + "where distributions are understood as row vectors.\n", + "\n", + "Here's a visualization for the case $|S|=3$, so that $\\dD$ is the unit\n", + "simplex in $\\RR^3$.\n", + "\n", + "The initial condition is `` (0, 0, 1)`` and the Markov matrix is" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "P = ((0.9, 0.1, 0.0),\n", + " (0.4, 0.4, 0.2),\n", + " (0.1, 0.1, 0.8))" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "tags": [ + "hide-input" + ] + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAAFUCAYAAACp7gyoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3gUdf4H8PfM7GbTAwYiRQ4VBBU0KogoKCBSBAQSkaYgRUFFUBTl4JATpamHBWy/Q8VTTjxP9OROqhRBRECkqTQbRaqEkLZ1Zn5/hA2bzSbZJLs77f16Hp47k+zuJ9ndee+3C6qqgoiIiComal0AERGRETAwiYiIwsDAJCIiCgMDk4iIKAwMTCIiojAwMImIiMJgq+T7XHNCRERWI4T6IluYREREYWBgEhERhYGBSWQwkyZNwssvv6x1GWW0adMGP/zwg9ZlEEUNA5PIQE6dOoX33nsPo0ePBgDk5OQgKysLSUlJaNy4MT744IOw7qe6t3v11VfRunVrOBwODBs2rNT3JkyYgKlTp1bp9yEyksom/RCRjrz77rvo0aMHEhISAABjxoxBXFwcTpw4gR07dqBnz57IzMxEixYtKryf6t6uQYMGmDJlClasWAGn01nqe71798YDDzyAY8eOoX79+jX7RYl0iC1MIgNZtmwZOnToAAAoLCzE4sWL8eyzzyI5ORnt27dH79698f7771d4H9W9HQBkZ2ejb9++SE9PL/O9+Ph4tGrVCitXrqzeL0ekcwxMIgPZvXs3mjdvDgDYv38/JElCs2bNSr6fmZlZ6ThidW8XjiuuuAI7d+6s8f0Q6REDk8hAcnNzkZKSAgAoKChAWlpaqe+npaUhPz+/wvuo7u3CkZKSgtzc3BrfD5EeMTCJDKR27dolwZacnIy8vLxS38/LyysJ1PJU93bhyM/PR61atWp8P0R6xMAkMpCrr74a+/fvBwA0a9YMPp8PBw4cKPn+zp07K524U93bhWPPnj3IzMys8f0Q6REDk8hAevTogS+//BIAkJSUhOzsbEydOhWFhYXYuHEjPvvsMwwZMqTk54cNG1Zm+Ud1bwcAPp8PLpcLsixDlmW4XC74fD4AgNvtxrZt29ClS5fI/+JEOsDAJDKQoUOHYunSpSVLOl5//XU4nU5kZGRg0KBBeOONN0q1FA8fPox27dqVuZ/q3m769OlISEjA7NmzsXDhQiQkJGD69OkAgCVLlqBjx45o0KBBpH9tIl0QVLXC/dW5+TqRzkyePBkZGRl49NFHK/w5j8eDzMxM7Nq1C3a7Pez7r+7tbrjhBrz99tto2bJl2Lch0qmQm68zMImIiErjaSVERETVxcAkIiIKAwOTiIgoDAxMIgtQVRWKomhdBpGh8bQSIhPyB6TP5yv5J4oi4uLiIEkSJEmCIISc10BE5eAsWSKTUBQFsizD6/XC6/XC/952nj6N3J9/RnxSEi7MzIQgSQAASZJgs9kgCALDk6g0LishMhNVVSHLMnw+HzweT6kuV1EUofh8+PaVV/Dz558D5wLRI4ro9MwzaNqhQ0mgCoJQ0uoURY7SEIGBSWRsgQHp9Xohy3LJ9/ytxMCW4rZ587Dj/fdRIAjIyclBYVER0hIS8KcGDXDXokWodcklJffrvw6IosguWyIGJpGxBI9Der3ekq+LolhhV6onPx8f9+mDfQcPonadOqiTnl58GokgoPDkSTTv2xc3TphQ5vEC/zew1cnwJIsJ+YLnpB8iHSlvHFIQhCoFV/7vv0MQBFzbqhUAwKcI+D0/EQ1TnbAnJeHkjh1lbuO/b0EQSlqzsiyzy5boHAYmkYZUVS3Vggweh6xuQMWlpECR5eLuVgh4c3dLbD9RB8+234oLPDlISE+v8PaBrdfAGtllS1bGj4tEMeQPH5fLhfz8fJw9exaFhYVwu90lXa2RCKSUhg1R58or4Tx9Gh/ub4Ycby1kX+vFaztawuX0onl2dtj3Fdi6VVUVXq8XLpcLHo8H8rlQJrICjmESRVHgOKTX6y05OzKccciayjt8GE/0nYM1v1+CR2/OQ4Low9vbUtH8UgfeXzu+Ro8bOFEosMuWS1TIJDiGSRQLwRN1qjsOWVOrv/VizdlWmPnni2E/fRD2pETMGJiJURM3YfXqw7jttj9V+77ZZUtWxBYmUQ2FGodUVTXmARno66+P4e67V+K113qjefM6AHAuxERs2nQIU6euxLffDkKdOgkRe0zOsiUT4bISokio6nrIWNu37wxuv/2/ePbZ29C2baOSr/sDEwDmzNmAU6fO4OOPe0alVnbZksHxPEyi6vAHpNvtRkFBAc6ePYuCggI4nU4oilKqG1Lr1tSxY4XIylqKRx65qVRYBhs79kb88ks+3n77h6jU4W9d+2f5+nw+uN1ueDwe+Hw+ThQiQ2ILkyiIv3VU3jikXltJeXkedOu2BLfe2hQjRrQq8/3AFiYA/PJLDoYN+zfWrbsTzZtfEPX62GVLBsIuWaLyBG4Y4PP5dDEOWRUej4x+/Zajbt00TJp0S8h6bTYbgq8D//rXLvz3v99jw4a7EBcnxahadtmS7rFLlsgveD1kXl4eCgsL4fF4ABRvGmCz2Qwx21NVVTz88HoANkyceHOV6u3f/ypccEEynn76m+gVGEJ5XbZut5tdtqRbbGGSJZS3HhLQdzdrOKZN24IvvjiKN97ojYQEe7k/F6qFCQA5OUXo3/8DvPtuF3TqVP64Z7QFd9n6P7QYoYVPpsMuWbIOf5efPxwDWy1GD8hAb731I155ZRcWLMhG7doVLxEpLzABYOPGg3jmmS+wdetApKdHbqlJdbHLljTGwCRzCx6H9C/3MMo4ZFUtXfobxo3bgLfeykKjRmmV/nxFgQkAL7zwJc6ezceHH96uq79VcHgapaucDI1jmGQulY1DSpJk2ovrli0nMGbMl3jxxdvDCstwjBvXDvv2ncW77/4YkfuLlOAPPP69bN1uN/eypZhiC5MMwz8O6T/6Sm8bBsTKTz+dRffuSzBlSifcfHPjsG9XWQuz+L5PY8SIj7F+fT9cdlntGlYaPcHjnYEfjKzwGqCoY5csGUvwAcr+5R4Aor5xuV6dOuXErbf+B/feex2ysq6s0m3DCUwAWLRoJ5Yt+xHr1/eD3R67pSbVFWq802azWe61QRHFwCT9Cz5AWVGUkgufGcchq6Kw0IsePf6L669vjAcfbFPl24cbmKqqYuzYJbj++nRMn35TNSrVBmfZUgQxMEl/Avdl9Xg8ZQ5Q5oWumM+nYMCAFUhOTsRTT3Wq1t8l3MAEgD/+KET//h/gn//sjltuaVjlx9Iau2yphhiYpD29b1yuR8UtvvX45ZcivPji7dXuJq1KYALA+vW/Ytastdi6dSBq146v1mPqAbtsqRoYmBR7ocYh/Rcwq45DVtWsWdvw2WcHMX9+XyQmlr8xQWWqGpgAMHv2OjidhfjnP7ub4nkKDE922VIFuKyEYsM/k7WoqAh5eXnIz8+H0+mEz+cr9QmfF6rKvf/+Xrz//j688krPGoVldT36aHvs3p2DhQv3xvyxoyFwiYqqqvB4PHC5XKXOMSUqD1uYVGOhDlD2YyhW36pVhzB69DrMn98XF19c8yUe1WlhAsC+facwatQn2LDhLjRpUqvGdehNebsK+fe5JUtilyxFRqhxSP/JHhyHjIzt208hO3sp5szpgczMehG5z+Djvapi4cLtWL16H9auvdMQS02qi122dA67ZKl6wjlA2T8LkReXmvvttzwMGLAcf/lLx4iFZU0NHnwN4uMdmDFjq9alRBW7bKkibGFSSIETdYxygLIZnD7twm23/Qd33XU1Bgy4KqL3XZMWJnB+qcmiRd3Rvr3xlppUF7tsLYldslQ+jkNqz+n0oVev/6FlywYYN+7GiN9/TQMTANat+wXPP78OW7cOQq1ajghVZhzBXbaBp6iQqTAw6Tyuh9QXWVZw992rIAh2PPPMbRDFyP/tIxGYADBjxhrIshvvvdfVsq+R4I0RAtd2WvVvYjIcw7SycMYhA7ua+KaPHVVV8cQTXyMnx4e//vXWqIRlJD322M3Yvv0UFi3ap3UpmvEHo/+94n9vud3uMj00ZB5sYZqUv+uI45D699JLO/DBBz9h/vy+SEmJXjdndZeVhLJ37ymMHv0JNm7sj0suiczxYmbALlvTYJes2QUfoOyf1WfWA5TN4KOPDmDq1C1YsCAbGRnJUX2sSAYmALz33nf48ssDWLPmTths7KwKxC5bw2OXrNlUdoBy4HIPvkn1Z926I5g4cRPmzu0V9bCMhnvuuRY2mx2zZpl7qUl1sMvWnNjCNJDAfVn9rUg/drMay/ffn8Ydd/wPs2d3Q+vWsVmiEekWJgCcPFmAgQMX4aOPeuDGG+tH9L7NiF22hsEuWaPxv7n84ejfuBxgQBrZkSMF6Nz5P3j00Xbo2rVpzB43GoEJAKtX/4yXXlqPrVsHITU1LuL3b0ahumw54U5XGJhGwHFIc8vNdaNLl8/Qq9cVuOeea2L62NEKTAB45pnVsNm8WLCga1Tu38zK2xiBH4g1xcDUo4rWQ/p3EuGbxhzcbhl9+nyOSy+ti8cfbx/zx49mYBYVeTFo0CJMndoGAwc2i8pjWAG7bHWDgakHHIe0JkVRMWzYFygqEjBrVldN1lpGMzAB4McfT+LBBz/F118PwMUXp0btcayAXbaaY2BqobwDlAEGpJVMnrwJmzadwmuv3QGHwxaTx1TdbkAUIdiLz9GMdmACwIIF32LTpl+walU2l5pECLtsNcHAjJXAcUj/FHL/C5ufEK3n9dd3Y/78PXj77SykpcVH/fF8Bw/CtWwZfAcPAQBszS5Dwu094LioIaIdmIqiYvToT9C160WYPPn6qD6WFbHLNmYYmNESOA7p8XjKbFwOcBzSqj799BdMnPg13nknG/Xrp0T98Xy//YaC+fMhSBKQlASoKtT8Agh2G9LGjYMt48Ko13D8eD4GDVqETz7phTZt9HE8mdmwyzbquHFBpARvGODfl9XlckFV1TKf/PgCtqavvz6Gxx7bgJdf7hmTsAQA19KlECQJQlIyBAgQBBFiaipUjweuNWtiUkO9eimYPLkT7r13JfLzPTF5TKsJtTGCx+MptTECz+6MPAZmGAI3Li8sLEReXh43LqcK7d17BvfcswrTp3dB8+Z1YvKYqssN3+EjxS3LIEJyMty7d8ekDgDo0uUyXHddQ4wfvz5mj2lV/uD092b5fL6SXYUC50xQzTEwy6EoCrxeL4qKipCXl4f8/Hw4nU74fL6SgXebzcaApDKOHStEdvZSPPLITWjbtlHsHtj/Mgx1gVQBxPjA4yef7ICvvjqGf//7QEwf18qC12t7vV64XC643W7IsszwrCEG5jmB3ax5eXml9mUNPmWdAUnlycvzIDt7GbKyWqBXr+YxfWzB4YCtSROoBQVlvqcWFMBxzbUxrScxMQ6zZnXDo49+iUOH8mP62FYX3GWrKAq7bCPAsoFZ1XFIosp4PDLuvnslWrSoh+HDr9OkhoQePSCIEpS8s1AVGarsg5KbCyE5CQmdOsa8npYt6+Gee67FiBGrIMvccFwL7LKNHMvMki1vPaQ/HDk5h2pCVVWMGrUWJ0/68MIL3SBJ2n0WlU+cgHvNGnh//BEQRdgzr0F8p06Iq1sH0V5WErIeWcGoUZ+gZ88/YeLE1jF/fCoreJat/2Qj9qCVsN6yksCA5AHKFE3Tpm3BF18cxRtv9EZCgl3rckKKxcYF5Tl+PB8DBy7CZ5/dgdato7+0hcLHjRFCsl5gFhQUlGw9x09OFC1vvfUjXnllFxYsyEbt2glal1MuLQMTAJYv348339yEzZsHIDmZp5roUajw9B98bTHWW4fpP+WD45AULZ9//htmz96GV1/tpeuw1IPu3Zvh6qvr4/HHN2hdCpUjeJatf54HZ9kWM3VgijGeRk/WsmXLCYwZ8yVefLEHLrooTetyDGHixI5Yt+53fPrpT1qXQhUInmWrqio8Hg9cLpelZ9maOlH8TzRRpB04kItBg1bg6ac7o0WLDK3LMYykpDjMnNkdY8euw5EjZZe/kP4Eh2fgLNvAuSFWYPrAJIq0kyeLkJ29DA89dANuvrmx1uVUgT7eD1dfXQ+DBl2D4cNXQVGsc7E1A6t32TIwiaqgoMCLfv2Wo3v3Zujb90qtyzGskSNbo6hIwUsvbde6FKoGq3bZmnqWrMfjQWFh4bnZgUQ14/MpGDBgBZKTE/HUU50M94HMZtPXcpejR/MwePCH+N//euPaa9mtbQYmmmVrvVmyAFuZFBmqquKRRzbA6QQmTerA11UENGiQiief7IChQ1egsNCrdTkUAaF2FTJTl62pA5MXNYqU2bO/w3ffncZzz3WD3S5pXU416e9i1aNHc1x++YV44gkuNTEbM3bZMjCJKvHee3uxcOF+vPJKTyQm6qtb0wwmT+6EVasOY8mSn7UuhaKgslm2imKcPYZNHZhENbVy5SE8/fQWzJvXC+npiVqXY0rJyQ7MnNkNY8asw9GjXGpiZhVtBC/LssbVVc7UgckWJtXE9u2nMGrUWvztb7ejceNaWpdjatdc0wD9+1+FkSO/4FITiwg+fsy/jameMTCJQvjttzwMGLAcU6Z0RGZmPa3LsYT77muD3Fwv5s7doXUpFENGOgyDgUkU5PRpJ7KylmL48Nbo1OlSrcuxDJtNxMyZ3fH8899i585TWpdDMeTf91vvTB2YfkacjUXaKCryon//FejQ4VL0799S63Is56KL0jBhQgcMGbICRUVcakL6YurANMInFtIPWVYwYsQaZGSkYsyYtlqXY1m9ejXHZZfVxZ//vFHrUohKMXVgAgxNCo+qqpgwYSNycnz4619vhSjydaMVQRAwefKt+Pzz3/D5579qXQ7FiBFOl9J/hTXEwKRwvPTSTnz11Qk8/7yRNyYwj9RUB2bM6IYHHliD48cLtS6HCIBFApNjmFSRf/3rAP7+9x8wd25PpKQ4tC6HzmnVqiGys1tyqQnphiUCk6g869YdwZ//vAlz5/ZCRkay1uWcpwLiycOQftwMcf92wFWkdUWaGD26Df74w43XXtupdSkURUZZVmLq00oAoLCwEF6vF5LEbjYqbffu0+jd+3+YPbsbWrduqHU557mdsK98H+Lx36AqKgRRgCpI8N2SBaXZddW+2+JTe/R/UQp2+HAu7rnnX1ixIgtXXVVH63IoChRFgcPh0NM4Jk8rIfI7fDgf/fotw5NP3qKvsARg2/AphGO/QU1MA1JqQ02qBcTFw/7lxxD/OKp1eTHXqFEtjB9/M4YMWQGnU/+7wZB5mT4wdfSJhXTizBk3srOXYfDga9C1a1OtyylFKMyD9Ov3QFIqEPhh79xZluL3mzSqTFt9+lyBiy9Ox+TJXGpiVkZo3FgiTTjph/xcLh8GDlyB669vhHvuydS6nDKEglyoogQIZd+aqt0B8bT1WphA8cV0ypRb8dlnv2L58t+0LocsyvSB6d/cl0hRVIwatRYpKYkYP76d1uWEpCalQpBlQC175JHg80JJs+4YXlpaPKZP74rRo9fgxAlrToIyIyM1aEwfmFxWQn5/+cs3OHLEhWee6azbjQnU5FpQLmoKFOWX/obsAxQFSssbtSlMJ66//iL07n0F7r//C76vTcQos2RNH5iAMfrGKbpee20Xli8/jDlzusPhsGldToV8He8C0upAKMiFUJAL5J+B4CyA74buUOpdXK37LM4Wc7wPHnywLY4dc+KNN3ZpXQpZjOmXlfh8PhQUFHBZiYV98snPmDhxExYsyEb9+ilalxMeRYF4eB/EEwehOhKhXNwSatoFNbpL27mJQ2Zw8OAZDBnyEVavzsaVV6ZrXQ7VgP+kEodDV5uGhPx0afrAlGUZ+fn5DEyL2rjxKO65ZxVee603mje37vgfYK7ABIBPPvkeH320Axs39kd8vL57Dah8RgpM03fJsjvWuvbuPYMhQ77A9OldLB+WZpSV1QING9bCX/5izaU2ZmKU67TpA5Os6dixQmRnL8Ujj9yEtm0baV0ORYEgCJg69TZ88slPWLXqoNblUDUZ5fBowAKBaZQngiInL8+DrKylyMpqgV69mmtdDkVRWlo8nn22K+6/fzVOnXJqXQ6ZnOkD049T0K3B45ExePBKtGxZH8OHV3/fVTKOG25ohB49LseoUav5PqeoMn1gGmV9D9Wcqqp46KEvIUlxePLJm/m8W8jDD9+IQ4cKMH/+91qXQtVglC1MjVFlDfHCaQ3Tpm3Bvn15mDHjNkiSJV7adI7dLmHWrNvx9NPfYO/eHK3LIZOyxFWFu/2Y3/z5P2Dx4l/x0ks9kJBgruUTFJ5LLqmNsWNvwpAhK+B2y1qXQyZkmcAk8/r889/w3HPf4dVXe6F27QStyyEN3XlnS2RkpOKpp7jUxEiMco1mYJKhbd58HGPGfIkXX+yBiy5K07oc0pggCPjrX2/DRx/tx+rVh7Quh0zGMoHJLlnzOXAgF4MHr8TTT3dGixYZWpdDOlG7dgKmTeuC++77AqdPc6kJRY4lAtMoM7AofCdPFiE7exkeeugG3HxzY63LIZ256abG6NatOUaPXsMPywZglF5AJgkZTkGBF3feuQzduzdD375Xal0O6dTYsTfi55/z8M47P2hdCpmE6TdfBwCPx4PCwkLYbNyg2eh8PgX9+y9HSkoynnqqo24/mYr5hyEeXgcx71fAlgi5XlvI9W8AJO1m8Jpt8/Vw/PzzaYwY8THWreuHZs1qa10OBVFVFaqqIj4+Xm/vZWtuvu6nsyeDqkFVVYwbtwFut4BJk27R7XMqnt4L+843IZ3ZBwg2wFcE26//g/2HdwDFp3V5ltKkSToeeuhGDB26Eh4Pl5rokZE2l7FEYBrlyaCKzZq1Ddu3n8bs2d1gt+v0uDZVge2nT6DaHFDjUgHRBkjxUB21IZ79DeIf3Ikm1vr3vwq1aiVi2rRvtC6FDI6BSYbwj3/sxcKF+/HKKz2RmKjfrkWx4CjgKwSk+NLfEASoUhykE9s0qcvK7wFBEPD0013w/vv7sG7dEa3LIQNjYJLurVx5CNOmbcGrr96B9PRErcupmOJDOcMfxV9nl6wm0tMTMW3abRgxYhVyclxal0MBjHR9tkRgknF9990pjBq1Fn/72+1o3LiW1uVUSkmuDwhSyGAUZA+UOi00qIoAoH37i9G5c1M8+CCXmugJA1NnjPSE0Hm//pqHAQOWY8qUjsjMrKd1OeGRHJD/dBsETx4gu4u/piqAOxeqIxVyBo8c09Ijj7TD3r25+Mc/9mhdCsFYh0cDFllWoqoqcnNzuazEQE6fdqJz588wYEAm+vdvqXU5VaMC0vHNkA6tLh7PVAHlgsshX3oH1HhtWsmCIECS+PoHgAMH/sDIkYuxYcNdaNpU/70WZqYoCmw2G+x23c1LCJnilgnMs2fPQhRFQ32asaqiIi969fofrr66IcaOvVHrcqpPVSB4CqBKdsCm7abwDMzSPvhgB1as2IMvv+yn3xnXFmC0wLRMlyyD0hhkWcGIEWtQr14tPPxwW63LqRlBhOpI1TwsqaxBgzKRkpKIZ57ZonUplmeka7MlAhMw1pNiVaqqYsKEjcjJ8WHq1E58zihqipea3IZ33/0RGzb8rnU5lmak97llAhMAZ8bp3Isv7sBXX53A88/reGMCMo06dZLw9NO3YfjwVThzhktNqHKWCUwjfYqxokWL9uOtt/Zg7txeSElxaF0OWcQtt1yCW265BGPGrOUHao0Y6dpsmcDkEV/6tXbtEUye/A1eeaUnMjKStC6HLGb8+Juxe3cOFi7cq3UppHOWSREeIq1Pu3efxogRqzF7dlc0aXKB1uWQBcXH2zBrVndMnPgVfv75rNblkI5ZKjBJXw4fzke/fsvw5JO3oHXrhlqXQxbWvHld3H9/G9x77wp4vTzVJJaMdG1mYJImzpxxIytrGQYPvgZduzbVupyKqQBUN6DyQmpmgwdfg/h4B2bM2Kp1KaRTlti4AADcbjeKioq4248OuFw+9OmzFE2aZODxx9tpXU6FRNdu2ApXQpD/AAQJcvy18CV1BcQUrUurEm5cEJ5TpwoxYMAH+PDD29GuXQOtyzE1HR8eDVh54wKAmxfohaKoGDVqLVJSEjF+/E1al1Mh0bkV9rx/AmoBVKkWVDEJonMb4s78H6A6tS6PoqBu3SQ89VRnDBu2Erm5bq3LMT2jXZctFZikvcmTN+HIEReeeaYzRFHHz4nqg61wKVQxGRASUPyBUwKkWhDk05Cc27WukKKkU6dL0a7dxRg3bp3WpZDOMDApZl59dRdWrjyCOXO6w+HQd/eg6DsKqF5AiCvzPVVwQHTv1qAqipXHHrsZ27adwgcf7NO6FNIRywQmaWvx4p/xyiu7MHduL6SmxmtdThhEoNzxfRV865hbQoIds2Z1x4QJ6/Hrr1xqEi1Ga8hY5l1vtCfGTDZuPIoJE77CK6/0RP36xpgso9gaAGJ88ezYUlRA9UCO57mWZnfFFRkYMeJ6DBu2Cj6fonU5pmS06zIDk6Jqz54cDBnyBaZP74LmzetoXU74BBG+5L6A4gSUAgAKoHogyGcAewMo8VdpXSHFwJAh10KSbJg9+1utSzEdox0eDVhoWYn/TExJ4qbesXL0aCE6d/4PHnzwBvTs2VzrcqpF9P4CqeALiN6DUMV4KAlt4Uu8GRCM0K18HpeVVN/JkwUYOHAR/v3vHmjbtr7W5ZiGjs/CBKx8gDTAQ6RjLS/Pg65dP8NttzXD8OHsvtQaA7NmVq/+CS+9tAFbtw5CamrZiWBUdUYMTHbJUsR5PDIGD16Jq65qgGHDrtW6HKIa69y5Kdq0aYRHHlmndSmmYrTrsmUCEzDek2NEqqrioYe+hCTF4Ykn2vNvTqYxYUIHfPPNCXz44X6tSzENo10fGJgUUU8/vQX79uVhxozbIEmWenmRySUmFi81efzx9Th4ME/rckgDlrqi8Yiv6Pr733/Ap5/+ipde6oGEBF2OSxDVSIsWF+Lee6/D8OGrIMtcalJTRmvEWC4wKTo+//w3PP/8d5g3rxdq107QupxyqQBk8RBctqVw2j6BV9oBFdwzlMJ3772tIJY2PhcAACAASURBVMsinn9+m9alUIxZZpYsABQWFsLn80EULfU5Ieo2bz6OAQNWYO7cXmjRIkPrcsqlAnDblsIr7QYgAKoECD4IajISvXdDVGtpXWLUcJZsZB0/no9Bgxbhk096oU2belqXY0iKosDhcOj1emztWbIAu2Sj4cCBXAwevBLTpnXWdVgCgCzuh1faBUFNgaCmQkASBDUNKgrhsi3VujwykHr1UjB5cifce+9K5Od7tC7HsIzW62epwNTpJxnDOnGiCFlZSzFmTFu0b99Y63Iq5ZG+A1Q7gl/2AlIgi0egCOadyGG0C5MRdOlyGa69tiEee2y91qVQjFguQdjCjIyCAi/69VuGHj0uR58+V2hdTlhUIR8CQnVLCoAqQgXPuKSqmTixA9avP4qPPz6gdSmGYtTrsKUCk7v8RIbXK2Po0FW49NK6GDWqtdblhE1SLipngo8MATD1GCZFR2JiHGbP7o5HHvkShw/na12OoRjt8GjAYoHJMcyaU1UV48ZtgNstYNKkWwz1grfLrQFBAEqFpgIV+bAprSDAoVVpZGAtW9bD3Xdfw6UmFmCpwAQ4llNTM2duw44dOZg9uxvsdmNtZC+pdZHgvRNAcfesinyoQgHsyjVw+G7RuDoyshEjWsPtVjFnzndal0JRZKllJT6fDwUFBTyxpJr+8Y+9eP757ViwIBvp6Ylal1NtKmQo4hGo8EJSL4SgGuOMzpoQRRGiyNd9NB07lodBgz7EkiV3oFWrC7UuR9f8R3s5HLrt1bH2aSUAIMsy8vPzGZjVsGLFITz44Dq89VYWGjfmWJ/RMDBjY/ny/XjzzU3YvHkAkpN5qkl5VFWFKIqIi9Pt34jrMKl6vvvuFEaPXou//e12Q4SlCsVcn/TIMLp3b4arrqqPCRM2aF2Krhnx8GjAYi1MRVGQl5fHFmYV/PprHrp2/QwTJ3ZAp06XaF1OhQrEUzgq/YBC8TQEiKgtN0IDuQXsqn636osVtjBjp7DQg/79/4nnnmuHrKymWpejSzo/CxNgC5MTfqrq9GknsrKWYuTI1roPy7PicRywb4BTyIVdTYCkxiFHPIh99rXwwqV1eWQhSUlxmDmzO8aOXYcjRwq0LociyFKB6celJZUrKvLirruWo2PHS3HXXS21LqdCKoDfbTsgqjbY4IAAASJExCERXjhxSvpF6xLJYjIz62PgwEyMGLEKisLrTShGbMBYKjCNuFBWC7KsYPjwNahXrxYefrit1uVUyiMUwo0iSCF28ZEQh1zpkAZVkdWNHHk9CgpkvPTSdq1L0SUjXostFZiAMZ+kWFJVFY8/vhFnzvgwdWon/r2IqslmEzFzZnfMmbMN27ef1LocigBLBia7ZMv34os7sHHjCbzwQnfDbEwQpybBgSTI8JX5ngwPast/0qAqIqBhw1Q8+WQHDB26AkVFXq3L0RUjfhi3ZGBSaIsW7cdbb+3B3Lm9DLWGTABwke8aKIIXXrigQoUCBR4UwY4E1JGbaF0iWVjPnpfj8ssvxBNPfKV1KVRDDEwCAKxdewSTJ3+DV17piYyMJK3LqbJU5UJc5u2ARPUCeAUnZMGDdOViNPd2gp17xJLGJk3qhBUrDuG//+UENCOz1DpMACgqKoLH4+FazAC7dv2BPn0+x3PPdUerVg20LqdCHsg4Jp6FR/AhVY1HhpICIWjJlAoVKPNVa+M6TO1t334Ujz/+OTZvHoAGDZK1LkdTiqIgPj5ezw0Ybo0HAE6nE263m4F5zuHD+ejc+TM89lg7dOmi70XWR8VcbLYdhCIoUKFCgIgU1YH23qZIVHW7AFoXGJj68MYb32DPnqP4/PM+EEXdhkXUGTUw2SVrYWfOuJGVtQx3332N7sOyQHDjG9tvsEFAohqHJNWBRNWOArjxjf0X832yI1O6//42yM31Yt68HVqXohkjT7q0XGCKomjoJyxSXC4fBgxYjjZt/oR77snUupxK/SblQIUKG0q3kuJhQ67gxFnBqVFlROGz2UTMmNEdzz33LXbuPKV1OZox6pp4ywUmwFamoqi4//61SEtLwvjxN2ldTljy4IQU4uUqQABUoEhwh7gVkf40apSGCRNuwZAhXGpiNJYLTKuHJQBMmrQJv//uwrRpnQ0zjpKC+JBnkKhQAQFIgHGWwRD16nU5mjatiz//eaPWpVAVMDAt5tVXd2HVqiOYM6c7HI6yW8np1cVyOiAAPiilvu5C8WzZWopxD7Qm6xEEAX/5y634/PPfsHTpr1qXE3NGvQ5bLjCtbPHinzF37i7MndsLqanxWpdToeOiglU2Nz61OfGt5IGkxuF6b2N4IaNI8KBQ8KBI8CIBdtzovYRLSMhwUlMdmDGjGx54YA2OHy/UupyYMmpgWm5ZiSzLyM/Pt9yykq++OoohQ1bhtdd6o3nzOlqXU6GvJDc22jyACkgAZAFwqAIGexORqio4KuXCBS9qqQmop6RBZFxWistK9OvVVzfh55+PY8mS3oYZIqkJRVEgSRLi4nQ9jMJlJYBxP9nUxJ49ORg69AvMmNFV92F5VJSx0eZBsiogFSKSICJVFSFDxad2Jxyw4VK5Dq6U66OBUothSYY3enQbnDzpxuuv79K6lJgx6nWYgWlyR48WIjt7GR599CbccMNFWpdTqZ2iFwJQJggTIeCsoOCYqIS+IYWkqioURYHP50Nxh5HpOo0Mz26XMGtWd8ycuQXff/+H1uVQBSwXmH5WWIt59qwb2dlLceedLdGzZ3OtywlLvqDCFvKpESCogAsMzMoEhqSiKCWvdVmWz/23AganvvzpT7UwfvzNGDJkJZzOsqfumI1RGy6WC0yjPlFV5fHIuPvuVbjqqgYYNuxarcupVL4MHPWIuMAjIdTKNBUqFAFIVy33kg2LqqqQZSUgFAFJkmCz2WC3x8Fms5e89hVFKfnH4NSPPn2uQOPGtTF58tdalxJ1Rr0OG2ddQQQZ9ckKl6KoePDBdZCkODzxRHtd/75FMvDpaTt+KBIhCoAPdrhTbBAzipB8bgKEChX5gormsg1pDMwS/pbkecK5yT0iBCHU36n471n8elChqgpkGRAE8dxkE/2+TqxAEARMmdIZ/fv/E927/wndul2sdUkUxJJXH7MfIj1t2hbs35+PGTNugyTp9ylWVWDBCTu+LxKRKgEpEpAmAmJePI4cTUYeFBRAQb6g4jLZhtt9CVqXrCl/QMqyXNLdWhyS/pakHZJkKycsgwkQBPHce+F8y5TjnNpKS4vH9OndMGrUGpw8WaR1ORTEsi1Mswbm3//+Az799Fe88042EhL0fYLHzy4Rv3tE1JJQ0rgRBSBDEpBb5ECHfBUXJMioo4q4wKItS1VVS/3z78EpSdK5/1/Tv4sQosVZ3Npkq1Mb119/Ee644wrcd98X+OyzO3TdQ1RdRv2dLHkVEkVz/tr/+9+veP757zBvXi/Urq3/1tght1A8hSf4vSMUtz5ltw3NFJvlwjLUpB1RFGGz2c79s0MUpQiEZaDzLc7iGoJbnRRLDz3UFseOOfHmm7u1LoUCWLKFCZhvluzmzcfx8MPrMXduL1x0UZrW5VTo0BkRG36VsPsPEX9AgK2+ipRaKBWcAoA4C+VkcUgGdocWtyLLH4+MlsrGOc//DEWP3S5h5szuGDr0I3To0BBXXpmudUkRxRamgRj1ySrPgQO5GDx4JaZN64wWLTK0LqdC23+X8OYmO348ISIegMcp4PBPIo4fQklW+FRAFIHLE8y9hKR4Zqtc0pITBJS0JKs2Hhktocc5uSwlNi6+uDYeeaQdhgxZAZfL/EtNjMCSgWmmLtkTJ4qQlbUUY8a0Rfv2jbUup0IuH/DpbhsS44BUB5BgA+onKBDigNMnReQVAHkyUCADvWr7kCKZ66Ic2NV6vrvz/KQdm00PIRlK6e5aLkuJnezsFmjQoBamTNmkdSkRYfSePb29M2PCLJN+Cgq86NdvGXr0uBx9+lyhdTkVUlVg634J+acAX8H5r19gAxo7FMSLCjy5wJUJCkbX8+LGVFm7YiMocGarLMsl45Hn10jaz03gMcJbsXgSkL9WjnNGnyAImDq1MxYv/gmrVh3SupyIMOrh0YAFN18HAI/Hg6KiIkNvwO71yujffwVSU5Px1FMddf0CPJsnYOGndhw4IiCnSIBNAOKSgIuuVmE/Nzcpzw1c21BBv6uNf6Cuf0ZrcTdr8fNSPBYpRGhmq56oJR8+Oc4ZPZs3H8aUKSuwdesg1K2r/wl95fG/VuLj9X1aErj5+nl6DpdwqKqKsWM3wOMRMGnSLbr+fRQFeOcjO46eFJCeBkgOAaID8DiBQ9sFqOeGKVUVaFbXuGOWle20I0m2KMxs1YPQ45xsdUbWDTc0wu23X47Ro1ebonfMqMz27rWEmTO3YefOHMye3Q12u35byaoKfPOtgp/2O+HOK4Ts9CLVoUBWBEh2wOsSkP8HcNYFpCepuCLDWN2w+p+0E0tclhJtY8feiN9+K8D8+d9rXUqN6PkDfmUs2SXr8/lQUFBgyC7Zd9/dgxde2IEFC7KRnp6odTnlUhRg8eIirP/Ki9yzAiQJEAQgMdUOx4VpyHWL8LkEpDRS0fZ6GX1b+pASr++XW6iu1vPb0ZmtqzUSQnXXGvdiqQe//JKDYcP+jbVr78Tll1+gdTlV5t98w+FwaF1KZdgl62fUTzgrVhzCM89sxbx5vXQdlgCwaZMTX399GnZbPiSbijiHBJtdRNFZL8T8IlxWV0VGioK7WnkxpLVXt2Fprkk7sRaqu1bmspQauPTSCzB27E0YMmQF3G5j9cgA5wPTqCz5LjfiE7Zt20mMHr0Wf/vb7WjcuJbW5VRo+/YTWLBgM3JyDuBMzgG4i76Dy/k7AECyizhzsghej4p4O9Cqhf7e9NrstGNmgcGpcllKDfXr1xIZGamYOtWYS02MeP314zveAH755SwGDlyBKVM6ITOzntblVOjnn3Px0Ud74fOJiIuLg91uR0K8DV7XEbicJ6GoArxewFmk4q6ePqQkaV1xMetO2oklf3ByWUpNCIKAv/71Nnz44X6sXm2OpSZGYcl3v5E+4fzxhxPZ2cswcmRrdOp0idblVGrNmoOw20XEx4vwnzxlswlISLBBVH9HfJyMC2oD40d5kXmFtq1LTtrREk9LqYnatRPwzDNdcN99X+D0aafW5VSJka6/wSx5NTg/k0/fb8yiIi/uums5OnZsgrvuaql1OWE5fDgPCQk2pKcXB6b/T2yziRBFLxIdbvTt7UCGBltjhtppRxCMsNOOmXH7veq66abG6Nq1GR54YI3ur2WBGJgGpPcnzedTMGzYatSvXwsPP3yD1uWELSkpDl6vgtRUEenpInw+wOsFPJ7iTbxbt47HzTfHboZc4KSdwPFIf3drcUiyq1V73H6vOsaNuwk//ZSHBQt+1LoUS7DsVULPgamqKiZM2IjcXBlTp3bSda3BbrqpAQoLi3frqVtXQtOmNtSrJyIlRcatt9bHwIEpiPZqHv94ZKhJO3Z7HCft6Bq336uKuDgbZs3qjr/85Wvs339G63JMz7JXDD3vJztnzg58/fUJvPBCd11vTBBK27YNcdllFyA314X8fDfcbi8ALy65JAn9+jWJ2uMGT9oByk7aYUAaTehlKRznLK1Jk3SMGdMWQ4euhMejv1nnwYzUAAhmyY0LAKCgoACyLOvu5JJFi/Zj2rStWLDgTmRk6GQKaRXJsoq9e//Ajh0n4fUquOqqumjZsi4cjsiGv7+71c+/V2vsz5Ck2AjcCKG4JcrNEIqpqopx45agVasLMGNGO63LKZeiKHA4HLq77oYQ8kVl2cAsLCyEz+fT1RO3Zs0RjBy5Bv/3f33QpInxdvGItlA77ZwPSO60Yy3cRSjY6dNF6N//A7z3Xld07HiR1uWEpCgK4uPjjdDK5E4/gfTWJbtr1x8YOXI1nnuuG8MyQDiTdjgeaUVclhIsPT0R06bdhpEjVyEnx6V1OaZk2auMnlqWhw7lo1+/5Zg48Ra0atVA63I0V3YTAU7aofJwWUqg9u0vxq23NsVDD63VVYMA0P8yvnDwiqOxnBwXsrOX4Z57rkGXLk21LkczwTNbAZQKSU7aoYpxWYrfI4+0w48/nsF77+3RupSQDNAdWy7LjmHq4RBpl8uH3r0/R9OmF+Lxx/U7UB8tnLRD0WXdcc4DB/7Affctxvr1d6FpU33sPW2gw6MBjmGWpvWnHEVRcd99a5GWloTx42/StJZYKW+nndKbCLAlSZFi3XHOyy6rg9Gjb8DQoSvg9ep/qYlR8MqkkUmTNuHoURemTet87pOvOXHSDmkvdHCavbt20KBMJCUl4Nlnt2hdSgmtGyo1ZdmrlJZP3Lx5u7Bq1RHMmdMdDodNszqihZN2SJ9Kj3OafRchQRDwzDNdsGDBj9iw4XetyzEFy16xtArMjz/+CfPm7cLcub2QmmqIvvywcNIOGcf57ffM3l1bp04Snn76Ngwfvgq5uW5NazH64dGAhSf9yLKM/Pz8mE762bDhKIYOXYXXXuuN5s3rxOxxo8G/iUDg68e/eYAochMBMprSuwj5/5llktCsWWvhdjuxcGE3zUJLUZSSo/MMgJN+AsX6RbNnTw6GDl2FGTO6GjYsQ41Hlp20w65WMiJzL0sZP/5m7Nx5Gv/8516tSzE0y7YwVVXF2bNnY9LCPHq0EJ07/wcPPdQWPXo0i/rjRVJ5LUluR0fmZ65lKfv2ncKoUZ9gw4b+aNIkLeaPzxamCUR794mzZ93IylqKO+9saZiwLDtpB5y0QxZkrnHO5s3r4r77rsewYSs1W2qipx3WqsPY1dfA+TGK6PF4ZAwevBJXX90Aw4ZdG9XHqqnikJRLHY91PiS5PpKszDzLUu6++1rExcVh5sytWpdiSJa+AkYzMBVFxYMProPd7sATT7TX3eywwPHI8yEpQBQlhiRRSMZfliKKAp59tiveeusHfP31Ua3LMRzLXw2j1SX79NNbcOBAPmbM6AJJ0sefubJJO8Uhya5WoopVtixF3+rWTcJTT3XGsGGrcPZs7JaaxKJXL9osfWWM1pP3979/j//851e8+GIPxMdruzFB4HZ05c1s5XgkUXWFCk5Z96eldOp0KW68sTHGjfsyZo/J00oMLhoD0P/97694/vntmDevF2rXToj4/Ycj1KSd861ITtohirzA4FQNsSzl8cdvxrffnsQHH+yL2WMavYVp2WUlAFBUVASPxxOxpSWbNx/HgAErMG9eL1x5ZUZE7jNcwSd/FI9H8uQPIu3of1nKnj0n8cADn2Ljxv645JLoLjVRFAXx8fFGCU0uKwkWySfuwIFcDB68EtOmdY5JWAZ2tZ7fjo6Tdoj0Q//LUq64IgPDh7fG8OGr4PMpld+gmszQHQswMCNyPydOFCEraynGjGmL9u0bR+Q+Qwme2Rq4qTkn7RDpVejg1Ms459Ch10EQbJg9+9uoP5ZBWpflsvSVVRTFGn/yyc/34M47l6FHj8vRp88VEarsvFCTdgJDkuORREZRdvs9Pcyu9S81efPNXfjmm2NRexyjhyVg8cAEavYker0yhg79Ak2b1sWoUa0jVlNlk3YkycaQJDIs/S1LufDCZEyZciuGDVuJvDyPJjUYgaWvuDUJS1VVMXbsBng8AiZN6lDjT0/BO+0IAnfaITK/0MtStBjn7Ny5Ka6/vhEefTQ6S03YwjS4mjyBM2Z8i127cjB7djfYbFX/MwZ2tYbaaaf45A+GJJE1lF6WotX2exMmdMCmTcfxr3/tj9ljGgmvxtWwYMEeLFr0E15+uScSE8Pfeb+8STvcaYeIivmDs/gaEOvu2sREO2bN6o7HHluPgwfzInrfbGEaXHWewOXLD+LZZ7di3rxeSE9PrPTnOWmHiKpHm2UpLVpciKFDr8Pw4asgy5FZauLfYczoLH2VruoTuG3bSTzwwDrMmXM7GjeuVe7PcdIOEUVO7Jel3HvvdfD5BDz//Lao3L9RWfqKXZXA/OWXsxg4cAWeeqoTrr66Xpnvc9IOEUVX2WUp0RrnlCQRM2Z0w6uv7sTWrccjet9Gxis4Kt+F4o8/nMjKWoqRI1ujY8dLSm4TPGlHEDhph4ii7fyyFCB645z16qVg0qROuPfelSgoqPlSE6MfHg1YPDDDaWEWFXlx113L0alTU/Tr16LCSTvFIcmuViKKleiOc3brdhkyMxti/Pj1Nb4vM7D8lb2i0PT5FNx77xeoX78WHnigFSftEJFOhQ7OSLQ6//znDli//igWLz4QmVINzNKnlQBAXl7x1GlBELBlyxZcd911EEURiqLgvff24tFHN6JJk1pISopDfLwNDsf5f8X/LSEuzoa4OFvQ96WAn5OCvi8hPv78bex20RQzyIhIL4ov3ZE6LWX37uMYN24JNm0agEaNUqpejaoiLi7OSN2yIf9Qlg/M/Pz8kmOxZs6ciWXLlmH06NEYMGAAVFXE4cNFcLsVuN0+OJ0ynE4fXC4fXC4ZLpcPTmfx14u/5kNRkXzuaz643XLJ9/23838t8D5kWT0XsFJJqAaGa3A4BwdycBgHBnVgwAfef1ycxJAmsoRQx4wBVQ3P+fO3YPv2Q1i+vC8kqWrBpygKHA4HA9Po8vLy4PV6z03YEZGbm4t58+bhiy++wHPPPYdbbrkl6jXIsgKXq3SQFv//suFbHNJyqZ8rKvKV/Lz/+4H/HXgbt7v4fz0eGXFxxQFa/K90K9gftsUBHRjOEhwOe6nwDQ52//0GtsSLb2cLeLMSUWwFBqdQ8i/c4JRlBffdtxh9+lyMJ55oVaVHZmCahM/ng8vlKllY6/936tQpuFwuNGrUSOsSo0JR1KCWcqjALhvOxa3owDCXg1rT52/ndpduiftDOrC16285h+rKPh/QZYM3sCUe/LXSrWyGNFFp57tr/YEZbnftsWN5GDToQyxZcgdatbow7Ec02OHRAAOzfP41lB6PB7IsB336okhRVTWgpSwHtYrLtpL9X3e75ZJWdHDr2f+14i7z0l/3t6btdrFMSBe3pkOHsf9nz3d/lx/GwQHvb6lXZ39hotgL1V1b8XVv+fL9ePPNTdiyZSCSksLbGpSBaUIMTvNRVRUej1ImTEO1noNb2IHhHNx6Dm51B3Z/u1wyRFEoCeTAYA0VvP7/HxjQocavg4M5+GftdknrPzcZVtXGOadMWYHatSW8+eatld+zWnzfDEyTYnBSTaiqCq+39Lh0ea3nUJPH/EEdavJY8IQz//93u+VzF6XSQVtxl3fooA4cvy4d+qXHp+Pji1vSfF+YSelxzvK6awsK3Bgw4AO88EJ79OnTpOJ7PHd/8fHx0Sg4WhiYVcXgJCPx+ZRyQ7ZsC7ts13bg2HTpVnXpFnXg2LR/hneocA4M29Ld3TbY7aG7xMubSFa6q5wzvKOv8mUpO3Ycw/jx/8XmzQPRsGFy+ffEwLQWBidRaLKslFpWVdkksODJZYFj02W7x8/f5nx3tw9erxIUyBV1d5cN69Bj0WW/VjrArRzS5Y9zvvnmZnz//REsW9a33Ml1/slFDocjVgVHAgOzpkIFp4GmSROZgn+Gd0Vd28HhXHryWPDEs7K384ezP/gDZ3jHx0tlQjawFRw4E7zsOunQYVzeRDJ9zfAuuyxFllWMHLkY/fpdisceuy70rRiY1sbgJLIWRTk/wzu49VzZuumiorJj06HHs32lZpG73cUzvEtvZlJeK7j0piZxcfaQ49flr50+31KvfFOC0sH5++/5uOeef2Hp0r645pq6ZX/63HaicXFxUXhmooaBGWkMTiKKlsAZ3pV1bQfP4A6eOFbeeutQY9OSJJQK47It4dI7hsXF2bBz53F4PB58993gMrO1FUUpOebQQEIGpi3WVZiJIAiw2WyQJAk+n4/BSUQRU9yNWbxlZq1asenO9M/wDl7fHGqGdmAQN2+eAEEAvF4vRBE4fPgwPvroI7Rr1w433nhjTGqPBd0E5qRJk3DhhRfi0Ucf1bqUUtq0aYMFCxagRYsW5f6MIAiw2+2w2WwMTiIyLEEQzrUaJaSlVe22J06cwDvvvIXFixcjISEB/fr1w+WXXx6dQjWii6v5qVOn8N5772H06NEAgFdffRWtW7eGw+HAsGHDwr6fnJwcZGVlISkpCY0bN8YHH3xQ49tNmDABU6dODet+/MGZmJhYskj3/Pl0RETm9eabb0JVVSxcuBBLlizBkCFDkJKSUrJXtxnoYgzzhRdewP79+zF//nwAwCeffAJRFLFixQo4nU68++67Yd3PoEGDoCgK3n77bezYsQM9e/bE119/XWHrsLLbuVwuNGjQAD/88APq169fpd9LVdWSFqeiKBBFLvImIvPy54l/dx9JkkrODjZYb1vIC7UufoNly5ahQ4cOJf+dnZ2Nvn37Ij09Pez7KCwsxOLFi/Hss88iOTkZ7du3R+/evfH+++/X6Hbx8fFo1aoVVq5cWeXfK7jFCaDkEGoiIrPwT4D096bFxcUhKSkJiYmJRjsHs0K6+C12796N5s2b1+g+9u/fD0mS0KxZs5KvZWZm4ocffqjx7a644grs3Lmz2rUFBqd/LRKDk4iMLFRIJiYmmi4kA+li0k9ubi5SUqp+ineggoICpAWNUqelpSE/P7/Gt0tJScGxY8dqVB9Q/uQgdtUSkRH4u1r9mxHExcWVdLda4Rqmi8CsXbt2pcFWmeTkZOTl5ZX6Wl5eXqVBHM7t8vPzUatWrRrVF4jBSURG4Q9IP/+YpCRZb7tAXbSZr776auzfv79G99GsWTP4fD4cOHCg5Gs7d+6sdMJPOLfbs2cPMjMza1RfKKHGOGVZZlctEWlKVVUoilLS5SpJEuLj45GUlIT4+HjYbDbLhSWgk8Ds0aMHvvzyy5L/9vl8cLlckGUZsizD5XLB5/OVfH/YsGFllpskJSUhOzsbU6dORWFhITZu3IjPPvsMQ4YMqdHt3G43tm3bdbwdyAAABbBJREFUhi5dukT+Fz+HwUlEWgsOSVEUER8fj+TkZCQkJFg2JAPpIjCHDh2KpUuXwul0AgCmT5+OhIQEzJ49GwsXLkRCQgKmT59e8vOHDx9Gu3btytzP66+/DqfTiYyMDAwaNAhvvPFGqZZidW63ZMkSdOzYEQ0aNIj0r10Gg5OIYsnf3aooSklIOhyOkhmudrvd8iEZSBfrMAFg8uTJyMjIqHSnH4/Hg8zMTOzatatKexNW93Y33HAD3n77bbRs2TLs20RK8TZVXni9Xq7jJKKI8YckAIiiWDKnwowzW6uJm68bFYOTiGrKH5L+o7kCNxTg9aQMBqbRMTiJqCqCl4EEtiR57aiQfnf6ofD41z0FboDAMU4iChQ4JqmqKmw2GxISEpCUlASHwxGT5SCTJk3Cyy+/HNXHqI42bdpUuplNRRiYBsTgJKJAeloGEnyYRnUPxYjGIRxVOUwjFF1sXEDV4w9Ou91e0lXLDRCIrCGwuxUAJEkq6XLV8v3/7rvvokePHkhISAAAjBkzBnFxcThx4kTJ4RaZmZmVrpFv0KABpkyZUnIIR7gqerzevXvjgQcewLFjx6p8mAbAFqYpsMVJZA1GWAYSeJhGdQ/FAKJzCEdNDtMAGJimEhyc/s2RGZxExmak00ACD9Oo7qEY1RXtwzTYJWtCwV21PI+TyHiCl4EYZaPzwMM0qnsoRnVF+zANBqaJMTiJjMUMp4EEHqZR3UMxqivah2nopx1PUeN/4/mnlbOrlkg/9LAMJJICD9Oo7qEY1RXtwzQYmBbC4CTSBz0tA4m0wMM0qnsoBhCdQzhqepgGA9OCGJxEsWeV00CCD9Oo7qEY0TiEo6aHaXBrPCrZcs/j8UBVVUONlxDpmf/66h+XlCSp1B6uZmWCwzS4lyxVLDA4AZTMziOiquFpIIbHwKTwMDiJqo6ngZgKA5OqhsFJVDGeBmJaDEyqHgYn0XmB+7cCgM1mg91uZ0iaCwOTaobBSVYVvNG5PySNtkaSwsbApMhQVRUejwderxcAg5PMSa+ngVBMMDApshicZDZWXQZCZTAwKToYnGR0XAZCQRiYFF0MTjISLgOhCjAwKTYURYHX62Vwku5wGQiFiYFJscXgJD3gMhCqBgYmaSMwOLlXLcUCl4FQDTEwSVv+4PSv42RwUiRxGQhFEAOT9IHBSZHCZSAUJQxM0hcGJ1UXl4FQlDEwSZ8YnBQOLgOhGGJgkr4xOCkYl4GQRhiYZAwMTmvjMhDSAQYmGQuD0zq4DIR0hoFJxsTgNCcuAyEdY2CSsSmKUmqvWgan8XAZCBkEA5PMwR+cPp+POwcZBJeBkMEwMMlcGJz6xmUgZGAMTDInBqd+cBkImQQDk8yNwakNLgMhE2JgkjUwOKOPy0DI5BiYZC0MzsjiMhCyEAYmWVNgcAI8yLoquAyELIqBSdbG4Awfl4GQxTEwiYDSGyD4Q5PByWUgRAEYmESBGJxcBkJUDgYmUShWC04uAyGqFAOTqCKyLJeMcZotOLkMhKhKGJhE4TBLcHIZCFG1MTCJqsKIwcllIEQRwcAkqg4jBCeXgRBFFAOTqCb0FpxcBkIUNQxMokjQMji5DIQoJhiYRJEUq+DkMhCimGNgEkVDNIKTy0CINMXAJIqmmgYnl4EQ6QYDkyja/BNxwg1OLgMh0iUGJlGs+IPT7XZDluUywcllIES6xsAkirXg4ATAZSBE+sfAJNKKPzh9Ph9Dkkj/GJhERERhCBmYHDAhIiIKAwOTiIgoDAxMIiKiMDAwiYiIwsDAJCIiCgMDk4iIKAwMTCIiojAwMImIiMLAwCQiIgoDA5OIiCgMDEwiIqIwMDCJiIjCwMAkIiIKAwOTiIgoDAxMIiKiMDAwiYiIwmCr5Ps8Ep6IiAhsYRIREYWFgUlERBQGBiYREVEYGJhERERhYGASERGFgYFJREQUhv8HCl4zbHW0g4wAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/kolmogorov_fwd_4_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "def unit_simplex(angle):\n", + " \n", + " fig = plt.figure(figsize=(8, 6))\n", + " ax = fig.add_subplot(111, projection='3d')\n", + "\n", + " vtx = [[0, 0, 1],\n", + " [0, 1, 0], \n", + " [1, 0, 0]]\n", + " \n", + " tri = Poly3DCollection([vtx], color='darkblue', alpha=0.3)\n", + " tri.set_facecolor([0.5, 0.5, 1])\n", + " ax.add_collection3d(tri)\n", + "\n", + " ax.set(xlim=(0, 1), ylim=(0, 1), zlim=(0, 1), \n", + " xticks=(1,), yticks=(1,), zticks=(1,))\n", + "\n", + " ax.set_xticklabels(['$(1, 0, 0)$'], fontsize=12)\n", + " ax.set_yticklabels(['$(0, 1, 0)$'], fontsize=12)\n", + " ax.set_zticklabels(['$(0, 0, 1)$'], fontsize=12)\n", + "\n", + " ax.xaxis.majorTicks[0].set_pad(15)\n", + " ax.yaxis.majorTicks[0].set_pad(15)\n", + " ax.zaxis.majorTicks[0].set_pad(35)\n", + "\n", + " ax.view_init(30, angle)\n", + "\n", + " # Move axis to origin\n", + " ax.xaxis._axinfo['juggled'] = (0, 0, 0)\n", + " ax.yaxis._axinfo['juggled'] = (1, 1, 1)\n", + " ax.zaxis._axinfo['juggled'] = (2, 2, 0)\n", + " \n", + " ax.grid(False)\n", + " \n", + " return ax\n", + "\n", + "\n", + "def convergence_plot(ψ, n=14, angle=50):\n", + "\n", + " ax = unit_simplex(angle)\n", + "\n", + " P = ((0.9, 0.1, 0.0),\n", + " (0.4, 0.4, 0.2),\n", + " (0.1, 0.1, 0.8))\n", + " \n", + " P = np.array(P)\n", + " colors = cm.jet_r(np.linspace(0.0, 1, n))\n", + "\n", + " x_vals, y_vals, z_vals = [], [], []\n", + " for t in range(n):\n", + " x_vals.append(ψ[0])\n", + " y_vals.append(ψ[1])\n", + " z_vals.append(ψ[2])\n", + " ψ = ψ @ P\n", + "\n", + " ax.scatter(x_vals, y_vals, z_vals, c=colors, s=50, alpha=0.7, depthshade=False)\n", + "\n", + " return ψ\n", + "\n", + "ψ = convergence_plot((0, 0, 1))\n", + "\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There's a sense in which a discrete time Markov chain \"is\" a homogeneous\n", + "linear difference equation in distribution space.\n", + "\n", + "To clarify this, suppose we \n", + "take $G$ to be a linear map from $\\dD$ to itself and\n", + "write down the difference equation \n", + "\n", + "$$\n", + " \\psi_{t+1} = G(\\psi_t)\n", + " \\quad \\text{with } \\psi_0 \\in \\dD \\text{ given}.\n", + "$$ (gdiff2)\n", + "\n", + "Because $G$ is a linear map from a finite dimensional space to itself, it can\n", + "be represented by a matrix.\n", + "\n", + "Moreover, a matrix $P$ is a Markov matrix if and only if $\\psi \\mapsto\n", + "\\psi P$ sends $\\dD$ into itself (check it if you haven't already).\n", + "\n", + "So, under the stated conditions, our difference equation {eq}`gdiff2` uniquely\n", + "identifies a Markov matrix, along with an initial condition $\\psi_0$.\n", + "\n", + "Together, these objects identify the joint distribution of a discrete time Markov chain, as {ref}`previously described `.\n", + "\n", + "\n", + "### Shifting to Continuous Time\n", + "\n", + "We have just argued that a discrete time Markov chain can be identified with a\n", + "linear difference equation evolving in $\\dD$.\n", + "\n", + "This strongly suggests that a continuous time Markov chain can be identified\n", + "with a linear ODE evolving in $\\dD$.\n", + "\n", + "This intuition is correct and important.\n", + "\n", + "The rest of the lecture maps out the main ideas.\n", + "\n", + "\n", + "\n", + "## ODEs in Distribution Space\n", + "\n", + "Consider linear differential equation given by \n", + "\n", + "$$\n", + " \\psi_t' = \\psi_t Q, \n", + " \\qquad \\psi_0 \\text{ a given element of } \\dD,\n", + "$$ (ode_mc)\n", + "\n", + "where \n", + "\n", + "* $Q$ is an $n \\times n$ matrix,\n", + "* distributions are again understood as row vectors, and\n", + "* derivatives are taken element by element, so that\n", + "\n", + "$$\n", + " \\psi_t' =\n", + " \\begin{pmatrix}\n", + " \\frac{d}{dt} \\psi_t(x_1) &\n", + " \\cdots &\n", + " \\frac{d}{dt} \\psi_t(x_n)\n", + " \\end{pmatrix}\n", + "$$\n", + "\n", + "### Solutions to Linear Vector ODEs\n", + "\n", + "Using the matrix exponential, the unique solution to the initial value problem\n", + "{eq}`ode_mc` is\n", + "\n", + "$$\n", + " \\psi_t = \\psi_0 P_t \n", + " \\quad \\text{where } P_t := e^{tQ}\n", + "$$ (cmc_sol)\n", + "\n", + "To check that {eq}`cmc_sol` is a solution, we use {eq}`expoderiv` again to get\n", + "\n", + "$$\n", + " \\frac{d}{d t} P_t = Q e^{tQ} = e^{tQ} Q \n", + "$$\n", + "\n", + "The first equality can be written as $P_t' = Q P_t$ and this is just \n", + "the {doc}`Kolmogorov backward equation `. \n", + "\n", + "The second equality can be written as \n", + "\n", + "$$\n", + " P_t' = P_t Q \n", + "$$\n", + "\n", + "and is called the **Kolmogorov forward equation**.\n", + "\n", + "Applying the Kolmogorov forward equation, we obtain\n", + "\n", + "$$\n", + " \\frac{d}{d t} \\psi_t \n", + " = \\frac{d}{d t} \\psi_0 P_t \n", + " = \\psi_0 \\frac{d}{d t} P_t \n", + " = \\psi_0 P_t Q\n", + " = \\psi_t Q\n", + "$$\n", + "\n", + "This confirms that {eq}`cmc_sol` solves {eq}`ode_mc`.\n", + "\n", + "\n", + "Here's an example of three distribution flows with dynamics generated by {eq}`ode_mc`, one starting from each vertex.\n", + "\n", + "The code uses {eq}`cmc_sol` with matrix $Q$ given by" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "Q = ((-3, 2, 1),\n", + " (3, -5, 2),\n", + " (4, 6, -10))" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "tags": [ + "hide-input" + ] + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAAFUCAYAAACp7gyoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeZwU1dnw/d85Vd09+7AIAoIooLijcQEVVEQQgdsIKioKAiqoaNQENzQoKovecTfxSWISjT6a547GN8ktCLgriWLcNwQVFWQflll6q+W8f1T3zDAOMEvPVFf3+X4+bY8zdPc1U1119bnOJpRSaJqmaZq2e9LvADRN0zQtCHTC1DRN07Qm0AlT0zRN05pAJ0xN0zRNawKdMDVN0zStCXTC1DRN07QmMPfwcz3nRNM0Tcs3orFv6hampmmapjWBTpiapmma1gQ6YWpawNx888088MADfofxI8cddxyfffaZ32FoWpvRCVPTAmTz5s38+c9/Zvr06QBs3bqVsWPHUlxcTO/evXn66aeb9DwtfdwjjzzCMcccQyQSYfLkyTv9bObMmcyePbtZv4+mBcmeBv1ompZFHn/8cUaNGkVhYSEAM2bMIBwOs3HjRj788ENGjx7NgAEDOPTQQ3f7PC19XI8ePbj11ltZvHgxsVhsp5+deeaZXH755axfv57u3bu37hfVtCykW5iaFiCLFi3i5JNPBqCmpobnnnuOO++8k5KSEgYPHsyZZ57Jk08+udvnaOnjAMaNG8dZZ51F586df/SzgoICjj76aJYsWdKyX07TspxOmJoWIJ988gn9+/cHYOXKlRiGwYEHHlj78wEDBuyxH7Glj2uKgw8+mI8++qjVz6Np2UgnTE0LkO3bt1NaWgpAdXU15eXlO/28vLycqqqq3T5HSx/XFKWlpWzfvr3Vz6Np2UgnTE0LkI4dO9YmtpKSEiorK3f6eWVlZW1C3ZWWPq4pqqqq6NChQ6ufR9OykU6YmhYgRxxxBCtXrgTgwAMPxLZtVq1aVfvzjz76aI8Dd1r6uKb44osvGDBgQKufR9OykU6YmhYgo0aN4vXXXweguLiYcePGMXv2bGpqali2bBl///vfmThxYu2/nzx58o+mf7T0cQC2bROPx3EcB8dxiMfj2LYNQCKR4L333mP48OGZ/8U1LQvohKlpATJp0iQWLlxYO6XjN7/5DbFYjK5du3LBBRfw6KOP7tRSXLNmDSeeeOKPnqelj7vrrrsoLCxkwYIFPPXUUxQWFnLXXXcB8I9//INTTjmFHj16ZPrX1rSsIJTa7frqevF1Tcsys2bNomvXrlx77bW7/XfJZJIBAwbw8ccfEwqFmvz8LX3cwIED+cMf/sBhhx3W5MdoWpZqdPF1nTA1TdM0bWd6txJN0zRNaymdMDVN0zStCXTC1DRN07Qm0AlT0/KAUgrXdf0OQ9MCTe9Womk5KJ0gbduuvUkpCYfDGIaBYRgI0ei4Bk3TdkGPktW0HOG6Lo7jYFkWlmVR/9yWUmKaJlLK2u8bhoFpmgghdPLUtJ3paSWalkuUUjiOg23bJJPJnUquUsofJUEhBJ988gn9+vWjtLS0NnEKIWpbnVLqXhpNYxcJU5dkNS0g6idIy7JwHKf2Z0KIRpPk6tWrefXVV3nttdf46quvGDBgALNnz6asrKz23yqldirb6pKtpjVOtzA1LUs17Ie0LKv2++nkuLukZlkWU6ZM4cQTT+TUU0/lkEMOwTCM3b5e/fv6rU6dPLU8o0uympbtdtUPmU6OLUlcicpKopVR1m12OWrgfk16jFJKl2y1fKYTpqZlm/rlUMuy9tgP2VyVa9aw6bPPuemBLbz+bg0v/X0ox57avG286idPXbLV8oROmJrmtz31Q2ZyxKqTTLLmjTe498kdvP95gpMG7s3SN9bw7+UXUVxW1KLY69/rkq2Ww/RasprW3tIJMpFIUF1dzY4dO6iuriYWi+G67k4ttkwnHte2efyvG3hl2XZ+cW4nTj3IoVOpyezb3m7R86WTeTpOx3FIJpMkEona1vEePoBrWqDphKlpGea6Lslkkmg0SmVlJVVVVcRiMRzHqU2Q6TmRbdky+98X1/HECzFuvqgrZSUmru1w2ahynn1uNS+99H2rnjudONN9mrZtk0gkSCaT2LatE6eWk3RJVtNaqbF+SKXULqd6tId//Ws9F164hF9O600XawNWPI4ZiVDUqRPrig7grrv/xX/+cwF77VWYsdfUJVsth+g+TE3LhPbsh2yJL7/cxhln/JM77zyNXs4aajZtJFRcDI6DFYux39ChPPrnL9m8eRvPPju6TWLd1Shbv/82mtZEug9T01rCz37I5lq/voaxYxdyzTUnMGhQL8JlZTiJBFXfr6Fm40aklLiWzdVXH88331Txhz981iZx6JKtlov0Sj+a1kC6dVS/zFq/teR3UtyVysok48YtYuzYQxkzpr/3TSkQpklpr54IV5GsqcGK1lASNlmw4AwmT/4rQ4b0oH//Tm0WV7pVmf67pueYZsuHDE1rKt3C1DS8gTqWZRGNRqmqqqKyspKamhqSyeSPJu5n48U9mXS46KKlHHJIN6ZM+Unt98PFxUilqF6zhuj69UgUrm0D0KdPJ2bMOJ5Jk5aQTDq7euqM0aNstaDTCVPLS+kWZDwe/1GChLrdPYIwQV8pxVVXvQGY3HjjkJ3iFaml8Er32Yfi7t1BCOyaaO3Px48/nE6dSrj99pZNNWmpXZVsE4mELtlqWUuXZLW8UH9dVsuysFOtLMjuMmtT3HHHu3zxxQ4effRMDGPnz8BmURFCSmp+WIc0JGZxCY5t1f5cCMHtt5/G+PFPM3z4vgwd2qu9w9+pZAvUlmzrb0kW1GOj5RadMLWcVL+/LN0XGYR+yOZ67LHPefbZb/jTn8ZRWBj60c8lgFIUddsbAbjJJHY0ijcA3vv9O3UqYs6c4VxyyUu8++75dO6cuakmzZE+HunkmZ7PqkfZatlCl2S1nNGwH3LHjh1Eo1EsywpEP2RzLVz4LQsWvMfDD4+hY8fGk5wRDmNGCoht2kR8SwWuZYFlQ4OS54kn9ua00/px5ZWvZkU5VJdstWykE6YWWHvqh0yvqBOEfsjmWr58IzNmvM59951Br17lu/6HhoFrW0Q6dSJSXoaQEte2ce0fD/L52c9O5Msvd/D445+3YeTN17AiYFkW8XicRCKB4zg6eWrtRpdktcBIl+nSfVxN2UA5F3311Q4mTFjCbbcN49BD997tv5XSIFRYSGzTJq9PMBxBOTbKdYCdS7iRiMn8+SOZOvVZBg/uwQEHdGzD36L5GpZslVK1H47qfzDKh/eA5g/dwtSyVv0FA2pqaqisrKSysjIrFwxoL5s3xxg7diGXX34cQ4b03uO/FyET5biYRUUYoRCOY6Nct3ZqSUP9+nXmiisGMWnSEiyr7aeatFTDKSr1S7b1581qWibphKlllfr9kPUTZLof0jTNvB05WVNjcc45izj99AMZO/aQJj1GCEGoIIJKeC2xUCSCsiyU1XjCBDj//CMoKytizpx3MhJ3W2tYXUiX6XXJVss0nTA1X9Xvh0wnyPoLBqRLbbnYD9kctu0yadJL7LdfZy6//NgmP05ICWYIIQVuMolVVYVynFRJdhePSU01eeKJL3jjjR8yEX67aNjqTJds4/G4XhhBywjdh6m1qz0tXJ6PLcc9UUpx7bVvEovB3LknN/vvY4RMEIJQYREiFEJZFk48sdvH7LVXMbfffhpTpy7l3XfPp2PHgtb8Cu2uYX9nempRerS0aZr6faY1m25ham2qsX7IbF24PFstWPA+//lPBXfffTqhkNHsx5tFRSjLwqqqJLllM8q2cJW7x8eddNL+nHxyH2bMyI6pJi3V2BQVXbLVWkInTC3jGvZDpjdQbvgJXyfIPXvyyRU8+eSXPPjgaIqKfrwwQVMIIZGhEOGOHSno0gVhmjg1NU167LXXDuaTT7by1FMrWvTa2UaXbLXW0CVZrdUa20A5rf4ne615li79nttuW87vf38We+1V1OLnCRUVoiwbq7oCBMhwpMk73RYUeFNNpk37Gyec0J2+fTu0OI5ssqeSbbrioWn16YSpNVtj/ZBKqR8NutBa7oMPNjNt2qvce+8o9tuvlfMhpYFhmpjlZQilcC0Lu6a6yQ/v378Ll112HBdfvIRXXz27RWXhbFZ/7mb95KnXstUa0h+htD1qygbK6dGs+uLSet9+W8l5573ILbecwoAB3Vr9fEZxEcp1SGzaTLKiAicWQzl77sOsb8KEIykoiDB37rutjieb6ZKttju6hak1Kr2zR5A2UM4FFRVxxo5dyOTJRzN0aJ+MPKdQChkKYRYUgOMCCjvatD7MNCkFd945gvHjn+a003oxePA+GYktW+mSrdYYfcQ1gNqdPWKxWO18yGg0GpgNlHNBLGYzfvyLnHRSH8477/CMPa9RUIhwXZwdO3CjUZRto1JLyjXHXnsVM3v2MKZMWcr27buflpJL9ELwWppOmHmq4cLlO3bsoKamhkQigVJqp+keOkG2PcdxmTLlZbp2LeOqqwZl9LmF8DaSNgoKEIb0Fi5wHHCbV5YFOOWUPpx44n5cffVreZkoGpZs0wvBJ5NJXbLNAzph5omm9EPqVqQ/lFJcf/2/2LrV5rbbTkXKzP7thWkihHeqpxOnSiRRTR0q28DPfz6EDz7YzDPPfJnJMAOl4QC39LmVXsvWbcGHES376T7MHJXezUH3Q2a/Bx74iDff3MDvf39Wm4xAFdJAhkysyh0IaeDGYoRKSlC2jQg3//UKC0PMn38G06f/jeOP787+++9me7E8sLtRtrpKk1t0CzOHNNxAueG6rLoFmX3+539W8dvffsZDD42mtDTSNi9iGijHJVRWjlkQIdyxI2483uyRsvUddFAXLrnkWC6+eAm2rVtTabpkm9t0wgywPW2gXH+6h06Q2ee119Zy443/5qGHxtC1a0mbvY4QEqMggr19G040hrV1G1KI3S7A3hQXXXQUphli/vzcnmrSErpkm5t0STZA0hsop0usdr09DXWZNVg+/bSCKVNeZsGC0+nbt1ObvpYwDXAcQmVl3jZfpuFt8bWLPTGbKj3V5Pzzn+G00/bl+OO7Zyji3KJLtrlDtzCzWDpB1l+4PL0uq+M4eqBOQK1dW83ZZy/ihhtO4phj2mM+o8AIh7FranBty0uaCpTd+g2iu3Yt4ZZbTmXy5CVUVjZ/qkq+2V3JVi8En/10wswyjfVDRqPR2g2UdYIMtu3bE4wdu5AJEwYwYkS/dnlNIUAa9VZicl1cy2rxKNmGhg3ry3HH7cs117yWkefLB42VbJPJ5E4lW508s49OmD7T/ZD5I5FwOP/8xRx7bC8uuujI9n3xkIkwTISUyEgEKSUqHs/Y08+ceRLvvLOJv/xlZcaeM1/samGEZDKpF0bIMroPs53pfsj85LqKyy57hZKSIq677sR2f30jUoCKRb1E6SYRptnkHUuaoqgoxPz5I7niiucZNKgb++1XlrknzyPplmd6WphlWViWpStLWUInzDZWP0Gmb3o+ZP659da3WbMmxq9//V8ZX5igKYQUyMIisC2kKUBKVCJzLUyAQw7pyuTJRzNlyhKWLh2HaeoCVks1XMvWcRwcx9mpW6b+YCKtfeh3dBtouIFyZWUlsVhM90Pmqd/85hMWLVrDvfeeQSTiz2dUGQp7CdJxUba3NJ5jtW6UbGMuvvholDK45573Mv7c+UqXbLOHTpgZUL8fMp0gG1swQPdD5p/nn/+GBx74iIcfHkN5eYFvcYhwCIFACm8QEI7T6mkljUlPNfnNbz5i+fINGX/+fKdH2fpLJ8wWaGzh8urqauLxeKMLl+skmZ/+9a/1/Pznb/LAA6Pp3r3U11iElMiQ128pVKrFmWybHUe6dStl1qyhXHzxEqqq9FSTtqBH2fpDJ8wmqL9weXo+pF64XNudFSu2cdFFS7nrruH077+X3+EgwxFIJkF5+2GqRLxFu5U01fDhB/CTn+zDdde90WavoXn09mPtRyfMXWjYD5leMKD+JrKmaeoEqf3I+vU1jBu3kGuuOYFBg3r5HQ7gtTC9XUsEAm9BdpVo2z0tb7jhZN56az1//euqNn0drU7DgYTpkm0ikdAl2wzQo2RT0q1I27ZrF0pOq//pTdN2p7Iyybhxixg79lDGjOnvdzh1TBOhvA+CKIUMhaEN+jDrKyoKM3/+6Vx11T8YOLAb++7rb1k6nzQcZeu67o/GVOjuoubL2yzQ3H5ITduTZNLhwguXcOih3Zgy5Sd+h7MTIxQC18WQEsM0IZnIyNJ4e3LYYd246KKjmDp1KU4rdkfRWk6XbDMnbxKm7ofU2pJSihkzXkfKMDfcMCQL3z8KGQ6Bq1DJJDIchjYa9NPQlClHk0zCr371fru8nrZrumTbOjldkq2/YEDDDZTTbxxNy4Q77niXFSsqefTRMzGMLHxfpVqYApChkNe6dNq+hQlgGJK5c0/n/POfYdiwXhxzzN7t8rrarumSbctk4ZmdOdFolGg0qjdQ1trUY499zrPPfsP994+isDDkdziNSr/nBcqbiKlclOOkRs22vW7dSrnpplO4+OIlVFfrqSbZZHcl2/oNDS3HE6ZSqjZR6gSptYUXXviWBQve45FHxtCxY6Hf4eyaUghpgFIQjSINs80H/TQ0cuSBHHFEd37xizfb9XW1pmtYsk2P89AlW09OJ0xdctXa0vLlG5kx43Xuu28UPXuW+x3O7pkmQnkjZEVBAdgWQilUG87FbMyNN57Ca6/9wPPPf9Wur6s1T8OFEZRSJJNJ4vF4Xi+MkNMZJX2gNS3TVq3azgUXLOb224dx6KFd/Q5nj2ovgODNwxTCS5Zu+54fxcVh5s0bydVXv8batdXt+tpayzRMnvlcss35hKlpmbZpU5Rx4xZx5ZUDGTKkt9/hNIkSwlusQIGqqQFAtGMfZn1HHNGNCy44kilTluK2c8LWWiffS7Y6YWpaM1RXW5xzzouMHHkgZ511iN/hNJkwDG+LL0AWFSFcBT6UZNMuueQYolGX++//wJfX11onX0u2OZ0wpZQ5edA0f9i2y8UXv8T++3dm+vRj/Q6neYT0SrG1N4XwcSEBw5DMm3c69933Ph98sMm3OLTWy6eSbU4nTNCtTC0zlFJcc82bxGJw880nB+59VRuuclHRKMp1vUFA7TxStr4ePcq44YaTmTRpMTU1lm9xaJnT2BSVXCrZ5nTCDNpFTcteCxa8z/vvV3D33acTChl+h9N8UkJq4XUZCiOFQCnafdBPQ6NG9eegg/bm+uv1VJNck4slW50wNW0P/vznFTz11EoefHA0RUXZuTDBnglk6nyQpuHti+k6Xn3WZ7NmDWXp0jX84x9f+x2K1gb2VLJ1fepHb4mcTpia1lpLlnzP7bcv5+GHx9C5c5Hf4bScTCVLBSQSSJS3H2Y7LY+3OyUlEebNO50ZM15j3To91SSX7W5VIScL3ot7ktMJU7cwtdb44IPNTJv2Kr/61Rn07t3B73Bap3ZdUOWVYR3HO/mz5NP9kUf2YPz4w7nkkpf0VJM8Ub/VmV73O9vphKlpjfj220rOO+9Fbr31FAYM6OZ3OBlRu3l0QQRhGJBl221deulxbN9u8dBDH/oditaO0iXbIFyvdcLUtAYqKmKMHbuQKVOOYejQPn6HkxnpZKkUwrK9+ZjgrS2bJUxTMm/eSO655z989NFmv8PR2lF63e9sl9MJMy2Io7E0f0SjFuPHL+bkk/swfvxhfoeTQQKhUgkzmYBEEgVZ0YdZX8+e5cyceTITJy4mGtVTTbTsktMJMwifWLTs4TguU6e+QteuZcyYMcjvcDIvPfDHNCFkIt3sSpZpY8b054ADunDTTcv8DkXTdpLTCRN00tSaRinFzJnL2LrV5rbbTkXK3HvfSCGQSkEohJQCkSUDfhoSQjBr1qm88MK3vPDCar/D0dpJEHaXyv4IW0knTK0p7r//I956ayP33BPQhQn2IH0aSKGQyQTScRCKrBkl21BZWYS5c0/n8stfYcOGGr/D0TQgTxKm7sPUduf//b9V/O53n/HQQ6MpLY34HU6bEeDth6lchGOnR/34G9RuHH30Powbd5ieaqJljbxImJq2K6+9tpabbvo3Dz00hq5dS/wOp00JmRrCLwVCSqQPW3s11/Tpx7FlS4Jf//ojv0PR2pCeVpIldAtT25VPPqlgypSXWbBgBH37dvI7nHYgECgIhcE0vCklWX5qhEIG8+ePZP78d/nkky1+h6O1kaBco/MiYWpaQ2vWVHHOOYu44YaTOOaYffwOp10Ib14JMplA2DbSEN6OJVmuV68OXHfdECZOXEwslv2rwWi5K+cTZhBGXmnta9u2BOPGLWLChCMZMaKf3+G0G6kUUimEAKlchFLeLiYB8NOfHsx++3Vm1iw91SRXBaFxE4yzpZWC0tzX2l48bnP++Ys59theXHTRAL/DaWcKgQDlgvA2kc6mlX52RwjBrbeeyt//vpoXX/zW73C0PJXzCTO9uK+mua5i2rRXKS0t4rrrTvQ7nHYnAIlChkPIkEHWd2A2UF5ewF13jWD69FfYuDHqdzhahgSpQZPzCVMP+tHSbrnlbdaujXPHHcNycmGCPRGuC0KBcrzpJSIrtsNslmOP7cmZZx7MZZe9pM/rHKJHyWaRIBwIrW39+tcf8+KLa7j33pFEIqbf4bQ7pQDpJUkcG2wbIaldLi9IrrhiEOvXx3j00Y/9DkXLMzmfMHWy1P72t6954IGPefjhMZSVFfgdji+8gT4KgUIKiTCEV5EN4PmRnmpy553L+fzzCr/D0TIgKNdpnTC1nLZs2Tp+8Yu3ePDB0XTvXup3OP5RgOt4XwiFwA30udG7d0euvfZEJk5cTDyup5po7UMnTC1nrVixjYkTX+Kuu4bTv/9efofjM+XdBKnRPwJwA9nCTBs79lD22acDt9zyb79D0VopKNfpnE+YWn5av76GceMWcs01JzBoUC+/w/GfSiXM+onTq8n6GVWrCCGYPfs0/va3r1i69Du/w9FaKCibR0MeJMygHAgtcyork4wdu5CxYw9lzJj+foeTPVw3lR9dby6mIQPdwgRvqsmdd47gssteZvPmmN/haDku5xNmmh6Cnh+SSYcJE5Zw2GHdmTLlJ36Hkz2U8pIkbu3IWAWBWelndwYO7MWoUQcxbdrL+jzX2lTwz5Y9CMr8Hq31lFJceeXrGEaYG24Yoo97fbUJk9qVfgjotJLGXHXV8Xz/fTW///2nfoeitUBQljANRpStpC+c+WHOnOV8+WUlc+eehmHkxVu76dIjZKkrywopciZhelNNzuD2299mxYqtfoej5ai8uKro1X5y3+9//xnPPbea++8fRWFhyO9wso9SgJNKkILaZX5E7lwC9t+/I1dffQITJy4mkXD8DkfLQblztuyGbmHmthde+Ja7736fRx4ZQ8eOhX6Hk51c17spBdJNnfkipxImwNlnH0bXrmX88pd6qkmQBOUanVtnyy4E5WBozffOOxuYMeN17rtvFD17lvsdTvZyHa/vUqrUyFgFhgDD8DuyjBJCcNttp/E//7OSl1/+3u9wtByTNwlTl2Rzz6pV25kwYQm33z6MQw/t6nc42c1xgFSZMt2fKUVOjJJtqGPHQubMGc6ll75ERYWeaqJlTu6dLY0Iyggsrek2bYoybtwirrxyIEOG9PY7nOynUiVZ8M56kWpp5ui5ccIJvTn99P5Mn/6K/rAcAEGpAubm2aLltOpqi7PPXsTIkQdy1lmH+B1OMLgO3ghZt+57Mvf6MOu7+urj+frrSv74x8/8DkXLEbl7ttQjpdSfMnOEbbtMmrSUPn26MH36sX6HExy23WB5PLzWpcytPsz6wmGT+fNH8stf/puVK7f5HY7WiKBdl/MiYUJwmvzariml+NnP3iSRENx880n6mDaDcGxv42ihUjcXTDNnS7Jpfft25sorj2fSpCUkk3qqSTYK0uIyuX22pATlYGi7N3/+e3zwQQULFpxOKJS7LaM24dpAqhwr8JJmKD/mq44ffzgdOhQxZ87bfoeiBZxOmFogPPHECp56aiUPPjiaoqL8uNBnkrAsvFGy9XYrMU1/g2onQghuv304Tz75Ja+9ttbvcLQA0wlTy3pLlnzPnDnLeeSR/6Jz5yK/wwkmO1FvHqbXb6TC+fPBo3PnIubMOY2pU5eydWvc73C0eoJ0fc6LhKkF1/vvb2batFf51a/OoHfvDn6HE1xWkp02kBZu3pRk0wYP3o9hw/pxxRV6qkk20QkzywTpgGh1Vq+u5LzzXuTWW09hwIBufocTbFYsNaVE1e1UEi7wOaj2d801J7JixXaeeOILv0PRCNbm0ZBHCVN/ogyWiooYY8cuZOrUYxg6tI/f4QRfIuZ1X6YXLQBUOOJrSH6IRLypJjffvIyvvtrudzhawORFwgSdNIMkGrU499wXOeWUPowff5jf4eQEYUXrtTBT50EetjABDjhgLy6/fCCTJi3GsvRUE63p8iJhBmmeT75zHJepU1+hW7cOXHXVIL/DyR3JeO0+mJDarSSSvzu7XHDBAEpLi7jjjuV+h5L3gnRtzouECcE6KPlKKcXMmcvYutVm9uyh+phlkLCi1LYuRaofMw9LsmneVJPTePzxz3nzzR/8DievBek8z5uECcFbhinf3Hffh7z11kbuuUcvTJBRCrASdYlSSDAlmPmbMAH22quY228/jSlTlrJtm55qou1Z3iTMIH2KyUfPPLOSxx77goceGkNpaX5fyDPOtcFNpuZg4vVlSgmhsN+R+e6kk/bnpJP2Z8aMV/UHap8E6dqcNwlTb/GVvV59dS2zZr3Ngw+OpmvXYr/DyT22hbCT9eZg4m0cnaeDfhq67rohfPLJVp56aoXfoWhZLm+yiB4lm50++aSCqVNfZsGCEfTt28nvcHKTk/BamULhbfGlwDTyviSbVlDgTTW58ca3+PrrHX6Ho2WxvEqYWnZZs6aKc85ZxA03nMQxx+zjdzi5K5kAZQEKDLzSrGFCSCfMtP79u3DZZcdx8cV6qkl7C9K1WSdMzRfbtiUYO3YREyYcyYgR/fwOJ6eJZA93oWwAACAASURBVBSh7LpdSsAb9GPk19J4ezJhwpEUFESYO/ddv0PRslReJUxdks0O8bjN+ecv5rjj9uWiiwb4HU7OE3YcpZydFy4IF+b8XpjNJaXgjjtG8Ic/fMayZev8DifnBfF6nDdnjF68IDu4rmLatFcpLS3iuutO8Duc/BCrBOy6jaOFQEX04KrGdOlSzC9/OYzJk5ewfXvC73ByXtCuy3mVMDX/zZr1b9aujXPHHcOQUh+TdhHfgRAKpAAhQLpQWOZ3VFlr6NA+nHjifvzsZ6/5HYqWZXTC1NrNI498zJIla7n33pFEIvmxeXFWiFem+i7TZVlQRaX+xpTlfv7zIbz33maefvpLv0PRskjeJEzNX8899zUPPvgxDz00hrIyPf+vPYn4dlR6STyJN1JWtzB3q7AwxPz5I5k58w1Wr9ZTTdpK0BoyeZMwg3ZgcsmyZeuYOfMtHnxwNN2765ZNu1IgrGqEdOtW+pESCvVx2JODD+7K1KnHMnnyUmzb9TucnBS067JOmFqb+uKLrUyc+BJ33TWc/v338juc/GMnUlt7CS9RShdMEyIlfkcWCBMnHoVhmCxY8B+/Q8k5Qds8GnTC1NrQunU1jBu3iGuvPYFBg3r5HU5+sqLeOrLpPkxILYunR8k2hZSCO+8cwW9/+wlvv73e73A0n+VNwkwL4tyfIKqsTDJu3ELOPvswRo/u73c4+StZjXATXjlWpvswTZ0wm6Fr1xJuuWUoF1+8hMrKpN/haD7Km4SpW5jtJ5l0mDBhCYcf3oPJk4/yO5y8JuM78OZgCm9bL4G3aIGpdyppjmHD+nHccb245prX/A4lpwTtupw3CROCd3CCSCnFlVe+jmGEuf76wfpv7rf4VhA24Hj9l8L1BvyIvDr1M2LmzJN5++2N/OUvK/0OJWcE7fqQV2dN0A5OEN1++3K+/LKSuXNPwzDy6u2VnaKbUQIwXBAOGAJVpHeFaYmiIm+qyS9+8QbffVfpdziaD/LqiqbXk21bv/vdZzz//Gruv38UhYV6Ye9sIBIVCOl4LUpDAgpVqkcrt9Shh+7NxRf/hClTluI4eqpJawWtEZN3CVNrGy+88C333PM+Dz88ho4dC/0ORwNvDmZiq3eWS9srx5oSinTCbI2LLz4ax5Hcc897foeitTOdMLVWe+edDcyY8Tr33TeKnj3L/Q5HS7NqkFaNN9AH8PbDDEGBPkatIaXgrrtG8Otff8Ty5Rv8DkdrR3mXMHVJNrNWrdrOhAlLmDNnGIce2tXvcLT6ktuAuDfYxwBMkVq0oKPfkQVet26lzJrlTTWpqtJTTVoqaI2YvEqYUu//l1EbN0YZO3YhM2YMYvDg3n6HozUg49tQJFNzLxXgQigCYb3KTyYMH34ARx21Dz//+Rt+h6K1k7zLILqFmRnV1RbnnLOIUaMO4qc/PdjvcLTG1KyjboeS1PZeBR1A6p1iMuXGG0/mjTfW8eyzq/wOJVCCeh3Oq4QppQxcCSAbWZbDpElL6dOnC9OmHeN3ONouiPgGRO3qPgKEiyru5ndYOaWoKMyCBSO55prXWbOmyu9wAiVom0dDniVM3YfZekopfvazN0kkBDfffFLg3vD5RCQ3eMvhGamdSgyBKtEJM9MOO6wbF154pJ5qkgfyKmFC8DqZs828ee/x4YdbWbDgdEIhw+9wtF2xYwh7W2pLL+HdmwYU7u13ZDlp6tRjSCQU9977vt+haG0orxKmTpat88QTK3j66VU8+OBoior0wgRZLVGBcOPe6FjD9Qb9SBMKu/gdWU4yDMncuafz4IMf8t57G/0OJxCCeD3WCVNrksWLv2fOnOU8/PAYOncu8jscbQ9kfB1ICwzHS5pSejuU6CklbaZ79zJuuukUJk1aQnW1nmqyJ0G8HudVwtRa5v33NzN9+qv86ldn0Lt3B7/D0ZpAxNZ4CVNSt+h6QWeQujLQlkaOPJDDD+/OzJlv+h1KVgvi5tGQZwkziAfIb6tXV3LeeS9y661DGTBADxgJChH/tt4emF4fplvc0++w8sJNN53CK6+s5fnnv/I7FC3DdMLUdqmiIsbYsQu55JJjGDp0f7/D0ZrKtRD25tR0Eryl8aSE4v38jStPFBeHmTdvJFdf/Rpr11b7HY6WQXmVMNP01JI9i0Ytzj33RU45pQ/nnnuY3+FozZHciKA61XfpeoN+QiEo2sfvyPLGgAHdOf/8AUyduhTX1debxgSxAZNXCTOIE2X94DguU6a8QrduHbjqqkF+h6M1k4x9B4blJUsJCFBGBCJ6rd/2dMklx1Jd7XD//R/4HUpWCuK1OK8SJgTzILUnpRS/+MUytm2zmT17qP57BZCIrwTDBpPUousKFdkbZMTv0PKKaUrmzRvJvfe+xwcfbPI7HC0D8jJh6pLsrt1334csW7aR//7vkXphgiBSIOxvU6Nj04N+gML9/I0rT+2zTxk33HAykyYtJhq1/A4nqwTxw3heJkytcc88s5LHHvuChx4aQ0lJ2O9wtJawtyJFRYOEaeAWHeB3ZHlr9OiDOOigvbn++rf8DkVrJZ0wNQBefXUts2a9zYMPjqZr12K/w9FaSFirQcbBdMFU3uhYMwwFvfwOLa/dfPNQFi/+nn/+8xu/Q9FaIS8Tpi7J7uzjj7cwderL3H336fTt28nvcLRWkIkvQNp1pVjDwTU7gNnZ79DyWmlphHnzTufKK19l3To91QSC2XjJy4Sp1VmzpopzznmRG288iaOP7uF3OFprKBBqhTfYx0yVYw1Qkf29Bdg1Xx11VA/OPfdwLrnkJT3VJKB0wsxj27YlGDt2ERdeeCTDh/fzOxyttdzNSLk1tdg6XtI0DFREb/CdLS677Di2b7d4+OEP/Q7FN0Gu8OVdwpRSBvqAZUo8bnPeeS9y3HH7ctFFA/wOR8sAaX0JMlq3YIEEzDBuWA/4yRamKZk7dyR33/0fPvpos9/h+Caoc+LzLmGCbmW6ruKyy16lvLyY6647we9wtAwRvJ8a7IPXujTBNTqCobf0yia9epUzc+ZJTJyop5oETd4lzHxPlgA33/xvfvghzpw5w5BS/z1ygptEsArXULiGizIACSp0iLeWrJZVxow5iH79unDTTcv8DkVrBp0w88wjj3zM0qVruffekUQipt/haJmiVuGGqnBNhWuCMsA1TBzzcL8j0xohhOCWW07lhRe+ZeHC1X6H0+6Ceh3Ou4SZz5577mseeuhjHnpoDGVlBX6Ho2WQEstxQgrHlN7NAGQhiAP9Dk3bhbKyCHPnns7ll7/Chg01fofTrnTCDIigHqjWeuutdcyc+RYPPDCa7t1L/Q5HyyCFi21+ii0ljiFxUve27A1God/habtx9NH7MHbsYVx6af5MNQnyoEudMPPAF19sZdKkl5g7dwT9++/ldzhahjl8jW1WY0uJLY1UwjRR8ji/Q9OaYPr049i0KcFvfvOx36G0m6Beh3XCzHHr1tUwbtwirr32BAYO7Ol3OFobSBr/xgYcTO8mDBxRAOJIlFK4rott24BK3bRsEgoZzJ8/knnzlvPpp1v8DkfbjbxLmGlBLgs01Y4dCcaNW8jZZx/G6NH9/Q5HawMuDpb8HBszdTNwkCi3F45dguu6te91x3FS/++iE2d22XffDlx33RAmTlxCLGb7HU6bC2rDJe8SZlAPVHMlkw4XXriUww/vweTJR/kdjtZGknyGLeKpZCmxkTiYCOc4DMPANE1CoTCmGap977uuW3vTiTN7/PSnB9O7d0dmzfqX36G0uaBeh/MuYUJwD1ZTua7iiitewzDCXH/94Jz/ffOVUopquYwEEhsDO1WSVRQQlgMxDBMh6p/iAhC131PKrW116sTpPyEEt946jL///WsWL/7W73C0RuRtwszlkuycOctZubKKuXNPwzDy8hDnpHR/pOM42LZNwt1GzFxbW451UuVY6fbBpHwPz+YlTu9caJg4c/fcyHbl5QXcddfpTJv2Cps2Rf0OR2sgL6+mudzi+t3vPuP551dz//2jKCwM+R2O1kr1k6TjOCilEEJgmibR8H9qB/t4LUwDF5OIe2IzXqHxxKlbnf459tie/Nd/Hcyll76Usx/sg3oNzsuEKWVu/tr/+7+rueee93n44TF07Kjn3wVV/ZGt6UE7UkpM00zdQiBhh/EhSUwsQlipkiyUEKElq/vUJU4vBl2u9dOVVw5i/foY/+f/fOJ3KFo9uZk5miDXPrm9884GrrrqDe67bxQ9e+6pHKdlG6UUjuPWG8nKToN2DMNESqO2/3E7nxDFwsbEwkhNKTGIuIdhEG5FJHX9nLpc659QyGDevJHcccc7fP55hd/hZJxuYQZIUA/WrqxatZ0JE5YwZ84wDj20q9/haE3kJUmnNiEJQW1LMhQKNTJop856418kMUgSqp1OoghT7p6UwQgbT5x6Wkr72G+/jlxzzYlMnLiYeDz3p5oEQV4mzFwqyW7cGGXs2IXMmDGIwYN7+x2Othv1S611rTaBlEZtqXV3STJtG19TJaIkCWFhYmHiYhJ29yHC3m0Q+c7lWj0tpf2MG3coPXp04NZb/+13KBkR9Mpe7mSOZsiVUbLV1RbnnLOIUaMO4qc/PdjvcLRGNDZoR0pZr9wawjCMPSbJ+lYbb5HAJEkYi1DtYJ/O7uA2/E1AT0tpf0IIZs8exnPPfcXSpd/7HU5GBHXzaMjjhBnUA5ZmWQ4TJy6lT58uTJt2jN/haPU0ZdBO/f7I5tjCaraJaGqgTwg7VZI1VQfKac+dSXQ/Z3vp0KGQO+8cwWWXvcTmzTG/w8lreZswg0wpxdVXv0kyKbj55pMC//vkguYO2mmpz423iREiQSQ14EfiYrK3czzCl9NZT0tpDwMH9uKMMw5i+vSXc6I6FlR5mTCDbt689/joo60sWHA6oZDhdzh5qzWDdlriO1ZSIWyShEmk+i8dQoQpoSuHZex1WkZPS2lrV199PN9+W83vf/+p36G0SpA/4OdlwgzyAXv88S94+ulVPPjgaIqK9MIE7SlTg3ZawsHmffNT4oRJEsauTZiSfe3jkGTLB6c9TUvRWiq9q8ltt73NihVb/Q4nL+mEGSCLF3/PHXe8y8MPj6Fz5yK/w8kLbTFopyXe4X22YxAnQoIwydRyeAWqjH04pE1fu+UaS5yOnpbSCn36dOLqq09g4sTFJBKO3+E0W3qlqqDSCTMg3ntvE9Onv8qvfnUGvXt38DucnNaWg3ZaYjPb+NTcSowCEkSwCOEQQmFwiHOsT32XzVE/cSo9LaWVzjnnMLp2LWP27GBONQni9Tct2880Dfjmmx2cf/5ibr11KAMGdPM7nJzUXoN2WuLvxmdUp5JlklBq/qVBuduBHuzf7vG0XDpx6mkprSGE4LbbTuMvf1nJyy/nxlSToMjLhBmkTzhbtsQYN24Rl1xyDEOHBunimP3ae9BOS/ydz9kowsQoJE4EizA2JhKDge4gX2NrHT0tpTU6dizkjjuGc+mlL1FREaypJkG6/jaU1wkz24dnR6MW5577Iqec0pdzz/V7FGTwNTZoR4j2GbTTEp9RwX9MlyhFxCkgQZgEYVwM+to96EgulOb18nstdcIJvRkx4kAuv/yVrL+W1acTZgBl+0GzbZfJk1+me/cOXHXVQL/DCaz6g3bq90emy61ekvSn1Lo7FcR4wtxOFUVEKSBJJLXAuqRQhRnMEX6HmGF6+b2W+NnPTuCrryr5058+9zuUvGD6HYBfsjlhKqWYOXMZ27c7PPTQ0KyONRt5SVKhlFv7t5NSIqXMusTYGBuLu+QOqlQJEoUpbBRJwKUAm9FOX4ysmUaSad7x8o6bSrU6QQiJlKL255onHDaZP38kU6Y8y+DBPTjwwI5+h5TTsv/q0UayeT3Ze+/9kH/9ayP//d8j9cIETdRw0A78eNBOEJIlwEwrzrpYGTXxEqJWAQk3TJIQCsmRdid60tnvENtJ49NSdD/nzvr27cyMGYOYNGkJyWT2TzUJcgMgGFeQNpCtB+2ZZ1by2GOf8+CDYygpac2+hrkvCIN2muvqCocfKspxd5RgVxeQSBSQdLwpJHu5IUYHalRspuw8LUUvv/dj48cfQYcORcyZ87bfoeS0YF1NMigbE+Yrr6xl1qy3eeihMXTtWux3OFlnV4N2du6PDF6STJv+JWzYEIatAqoFxMIQD+E4JmEFM9yefofoMz0tZVeEENx++3CefPJLXnttrd/h7FY2XnubKphXlgzItpLsxx9v4ZJLXubuu0+nb99OfoeTNZoyaMev+ZGZkkjAhW8Itv5gwBagEqgB4oBrYCjFdU4Hion4G2hW0dNSGurcuYg5c07jkkuWsnVr3O9wclJwrzKtlE2bSH//fRXnnPMiN954Ekcf3cPvcHz340UEVL1SazgnkmTayk1w0bOS5BoJm4AdeMkyBliAgitlhP3QSyE2Tk9LqW/w4P049dR+XHnlq1nVIIDsn8bXFMG/4gTc1q1xxo1bxEUXHcnw4f38Dsc36SSZXo4O2ClJBrnUuiv3/xNu+Z0B3wnYyM4tSxsQMLHI5nhDtyz3TE9LSbvmmhP5/PNt/PnPX/gdSqOCXJIVe8j6OftOSyaTRKNRDMO/UajxuM2ZZ75Av35784tfnOhbHH5Jl1vT0ht7B2X6R0t99x3MnANYBkSAEqA8desAdAH2gvMPdDg7XwbEtglV26rJt2kpq1Zt4dJLn+ONN86lX7/sWOAifSwKCgp8jqRJGn2j5G3CtCyLmpoa3xKm6yomTXqJREIwb96I1Mmc25RStUky/SmzLkGKnE6SAJWV1cyYsZl4XABFQBlQCCUSOuIlyq7ebcoxDqO6+xltLmkscUKuJ8+nn/6QxYu/4PXXz8mK6Wm5kDBz+wqVxW6++d+sWxdnzpxhOZ0s82HQTlPMnr2MSy55mXj8Y2AF8D1QAcQhileCVYCpmDVQJ8vMaryfM9fLtRdcMIDi4kLuvHO536HUCnI5FvK4hWnbNtXV1b60MB9++GP+9KcV/OEPYykrC8SnrWZJr7RT/+0TpJV2Mu2xxz5i8eJvUv/nrWADhUAvotFSamo2U4WFKitl6IHlGLIURSGqIIwsBVkGshTMIjBKwSyBSDFECrxbSQQKCqAgBMUhRYnhUIJDCS7FCIoxCKE3G6/jvS/zoVy7ZUsN48c/zdNPj2TIkH18jSW9F2YkEog+eV2Src9xHKqqqto9YT777FfMmvU2f/zjOLp3L23X125LP16OTiBl7vdHNsW0aS+ybVuMupUoHZSSrF8fpbp6LeDinZ8lwD5Az9R9JyDilWo7p24dUvdlqa87pL4uB0pTX5coKE1iFsUoLI5SEo5TJOIUqzilKkGRSlLq2pS6SUpth3LboNwNUWqFKVEhSqxCInYhxXYxEbuEMMU5vBRf7pdr33hjNfPnv8p//nMBHTr4l6xc18UwDMLhQCzI0ugbIG/XkvXDm2+u4/rrl/HrX58Z+GSZ7o+s/4HL6480kTKY/ZGu6/Dpp5V8/PFGTFMxcGA39t+/9WtzhkIN/xaKmpoaqqvX4X0mDeHVZKuBrXiJswSvFRqCGul1eUaBgnr3iXq3OBBOfR0SEAphmw6JsIMwFK5p48oQSSyiSGpw2IZNAfHULUmYBAUkKCBJoUoSxiGiEhQqh0LXIeKEiDghCq1iCq0OFLp7YVqdCCU7Ip0uSLsDZuCmv4hG1q0VtbdcSJwnnbQ/y5Z9y4wZr/LUU6f7WhYNekk2bxNmex+4L77YyqRJS5k7dwT9++/Vrq+dKbtOktk/aKciBp9vgVUb4ft1sHEtbF0jcNcJiCbxEtUPwHogCSj+9reVjB3blwsuaN3Wamed1Y/HHvsE17VT3xFYliTVaZn6nsRradqAk7ql/s5uvVv6T19/jn5jXyO851TpY2KgEKnXs3CRKCQKgar9mURh4CCxhEDiIIVCYqOMJHaoiiQWCSyieAk1rCxCyiKsXEwlCLkmYauEkN0Rw+qOTPZCJXuA3R3X7gaqE9k5dKIucQL1BqaJnCjXXnfdEC644Bn+7/9dwUUXHex3OIGlE2Y7WLeuhnHjFvHznw9m4MBgLW8WhCRZSYK1JPkOxbeOwQ8xk/XVEaqrJW6l8BYDqAS24329HS8/VgBxh7qW3Xa8ZAlgopTD//7vNwwZ0ouePctbHN/w4X0oKDB5/vmvqKmxOfLIHhxzTClXXvkDyaSNdxqmp9eY9W6pC7XEuxl130I2+Fo2+D4KhEII73lFKjWK2tTofa+xROA9raj9F94t/cjUXrIIFApXKFwBDg4GFq5K4oQ3IZSNFBZS2ZiuSxgwlYFhF0CiAyrWAztxILbVj6TTG9vticPeeM1kP+XmbikFBSYLFoxk2rS/cfzxPejbt+Xv53yWtwkzLd0R3VZ27EgwduxCzj77MEaNOrDNXieTfjxoR/g2aMfBpoZqtlHFVmrYTJzN0marEFQKk2oi1BAhqgqJJQtJxkNQXQA1om7FnDheqdLCy4dJvIacTep3TLfo0i1AWXtv2y4VFbFWJUyAIUP2ZciQfWv/XynF2LGH8de/flyv5VmM12/ZCa9zsghMUdc/ma7Ulqb+aTFe1bYYbz5nQfqmIGwhwgmkYREykpjCwsDGwEKSROIgcJDYqa8VCjf1d6i7T6eIndMnO92LVN717gVSpXN26rwSDq6wQTmI0DYMYw2h8McYiUUYlgF2GDcZQVlF2E43EtZ+JNwDsDgQi32xxL4ooyO0ezmvsXJtcPs5+/fvwqWXHsvkyUt45ZVxvkw1yaYV1loibxNmXR9F20kmHSZMWMIRR/Rg8uSj2vS1WqvhIgLtlSQdHCy2E6eCGiqoETuoEpVUiwTVQhLHII5JnAhxQiSJkCBCnAISmMQIYRHGUSFc1wDHINXkqbvup0uVDWcRiPR/0s239Ongpv7fxTQN9t67JOO/txCCWbNGcPbZR/D669+yY4eirKwTHTt2oLy8iKKiIopLDSJFLoaZGiFrgKO8KqsSYBuQFGAJiEuwQhAPuSQMm6Tj4kQFdsLErY5gS1BCYhkCWwJSoAwXKRyE4SCFJCTBRHhdoAhCCEzhInG98myqiAsq1UJ1vDG/wkUoN9UYVnV/YuFS24hVyvuJEqmbAqlAOggjimlEUWoHJuspcD8E10DZBsopwHWKcFRHLLc3CQ4kaRxEItwP2+yJa3SENr8IN97PGcRy7YUXHsWyZd8xb9673HbbIL/DCZy8TZjQtmVZ11VcccVrhEIRrr9+cNZ1djdWaq1LkpkttdpUo9iIwwZsNhGTm0mKbSREggSKJJIkJklCqaEnXhL0/t9rFzmp/jevv82rQSoMRP3+sNoWzU6/0s5ly3ReNPFaZZbEa5aV4A0xTY+gsRFCMmHCwXTrlvmEmda/fzf69++WwWeU4IS9DwqJ5k9ZsrCwRIyEWU3crMQ1KnHMKhxjO7ZZAeZWMLZCqJKwqEHKGkzh1JV4BQjhtV6FApFqpcpUkRfRoMNVCUSqn1XiotJ9rkIhhIMSNUg3iulUEHZWUWK/grJNFAW4dhG26kRS9iFeeAhW5BCSBftjhXt4nzAybud+ziCWa6UU3HnnCM4772mGD9+XE07Qa1c3R14nTGi7kuztty9n1aoqHn30pxhGdpQh2qw/0nXA3QHOBlDrEOoHhFyLbVZgmdVYhouFTBUETWxMkql7ixBOqv3ipi67df1not4lqOExUohUszHdN4dQIF0wXAhJb/BpBK8Ua6fu0y1NqEugNYXgdqEwUkxxcSe6dKnhiCNMRozYl7KyoI36bJ0QIUIqRJFVBtaeL6YW1bjGVpS5GRHehDA3YoR+wAyvxZAbMEJbkbIKIeJI4kjHRSoH4SqkIxFKgCsRSgHSS6gC0gdJKJn6HKSQQqKEQggbYdcgnBqks4WIu4qS6qUI28RRxbiqGCvUi3jBwSRKDsEqOQCrsDduqDxDZd1g93N26VLML385jMmTl/Luu+dTXt4+U03ao6rX1vJ2HiZAVVUVrutmvK7+u999yiOPfMof/ziOjh0LM/rczZWx5eiUC8kdkNgI1nqktRbsH5BUgNwORhxMB0IubsjFNQVWSGIbJrYwapOjTQivN837f6v2Ppy6D5FIfS9JJHXvtTYThGu/TlJAkhBxIqkybZi4U0AyGcJKFEAshBk3KIorOtiKzgr2BvoWQ78y6Nn62SJak+1AmhsxzDWEQmsJidUUiG8Ii7WYqgLDjSIcC2nbCMvGdUDYAtcGYYNwBMoR4CiELcARKFsibIVSXvld2UZqgLFE2d6/9erQBq4qwJVlWEY3EiUHEu9wFMmyftjFvXEjmUqi9edzBmNayl13vQIkeeKJEe3yeq7rEolEgtKPqRcuaKimpgbbtjN6AP/5z9Vcd91b/OEPY1s9UKSlGh+008RFBJI1EN8M0Y2I6A+I+A+I5AaEu81LitICwwHTBUN5g1JCymvNmQpMcEMKx5TYIQNbGjiphJn8UcI0sQmlvu/93KqXWJOpm0UYmxB2KmG6REBFMFUhRaqIclVABwroRgHdCVGq940MEBuDzYT4AVN9T0h9RcT5mrDzHYazBenUIJwEwrYRjguWRDkuwpFecnRB2A7KMRCOi+sYSFugHMAW4AqU43X+CkegXMA2cEQhjlGOHelGosMhxDsNINnxQOzSnqhwazZv3zlxZnM/ZyxmccEFz3DLLccyYUL/Nn8913UpKCgISitTJ8yGotEoyWQyY6v9vPPOBs47bzEPPzyGQw7pmpHnbKrGB+3sIkk6DsS2Q/VGqNqIrPoeUbMOkagAtxpk3EuK6fKmiVe+DJFKktQmR4zU90Mq9bXCNcENCSzTwJEmlpSphJdOlAZOKkmmW5UOJjYRHEI4FCDdEsJ0IOJ2IkwpZXSilFJKKCEbLz5aG3ASGO4GQs73hK3VRJIrCFvfEEqsxbCrwEkg7YTXmrQEKFCOBAeErcARqamtXtk3PRhaipPzGgAAGrxJREFUOSa4CuVIr0vVEShMXIqwIx2winsT73IE8a5HYHXYD6d4bzBa0nuV/bulfPHFJi6//HmWLRvP/vu37Qd8nTADLhaLkUgkMpIwV63azsiR/2D27FMZPLh3BqLbvfqlVqj/abbeoB3bguoK2LERsW09YsdaROUGRKwCVAxUEqRTlxQN10t66Zvpei1Ig3r/hrqEWX8Ajalqk6gKgWsI7JCBY0gsw2tJOpg4RHApRqlOoPbCdLth0IUQnYmwFyHaboCNliNcG2lvImx9Ryj2DQXVXxCKrSKcWI+0qpF2FGwH4SiULcFWgNcCVbZEuF4yVa5IJVUFTghwcR2JcL0E6xLGNYuxI12JdzmEeLcBJPc+CLu8FyrSnL7t7F5+7/HH32PZsq956aVxmGbblEvT1yudMAMsHo8Tj8dbnTA3bowybNj/x9Spx/DTn7bdKhr1B+2kBysJIZCOjajahti2Gbl1HWxZi9i+ARHbBk4MXMtLeNL1hvIbqi7hSVWXHA3ASCfIdAtS1bUwzVQJtv7jDVJlWQGGBDMEZgGuUYIb6oIj98GV3UD0APbGoAMyZ9cl1XzlxDDj6wnFVhOu/JLC6s8JVa/GjFcgEzGwLYSrULZAKq98KxzXm4bkSqj92kS5LjjeqDDlKITrTapxZDFOQUeSHfsQ32cAiR6HY+21P25RU/pCs7Of03UVl1/+PMOG9eDWW49rk9dIX7MKC/0d09EMOmE2lEwmqampwTRbPli4qirJGWf8kxNO2J/p04/NYHSenQbt2A5UbcfYthm5ZQNy0w+IresR1dvBjoNKgHLqEqOkrlUoXZCpZGeouhKrJPW91L1UXvKTNGhVytS9CaEQhApRZkdUQTfcgh4Q6ubdzK5g5NfIUi2LuQ4yvplw7DvC21ZQsP1zIttWYkS3IKwo0rLA8fpApeN6g4hc6SVShTcf1FHgSFwlEcprobquAAyUjOBEykl23I94zwHEex2G1bUvbmnn3STQ7NstZePGas4//2mefXY0gwZlfm+5gO2FCTph/lgymSQajba4hWlZDuPHL6a8vIRbbz0lY6UGx3Fgxw7YsgU2b8BYvxZj80ZkZQUiGQMnTu38iNol09y6xGiIVMJMJUejXgI0ZKq1Sb2Wpaz37wSETDBDqHAhFHZClXRDFXVHFXWDwu4Q7gxmYD4patrOlIsR20Jox3eEd3xJ4abPCG39ErOmApms8boylDcKF1JlW9tNLYIhvUUXlMRVBkKlSrs2oATKKMApLCfZoRfx3keR2O9wEl37oMo67SKBZk8/58svf8UDD7zJ8uUXUFaW2SUKdcLMAZZlUVNT06KEqZTiiiteZ82aGPfdN6rFtX83FsPduhV7wwbctT8g1/2A3LIZI1aNtBIYyvJmJ0oXYQgg1eoTqm4yfm1ypO57QtW1INM/N2XqXnjLxoRNCBWgSjqiyvZGlfVAlXeH4i5Q0AUiuj9RyxOOjRHdRGj7NxRs+pzIhk8JbfsWs2Y7MhFDuK438tZNrVTkeEsC1n0PUKlFNVzpDURC4BoFuIXlJDv1JL7/kcT7HInVbX/ckoYl3J3LtXUj99s3ed5xx0uEQg5//OPwjD5vwPbCBJ0wf6w1m0jfdde7LFy4ht/+9iyKiva8Oa9yXZwdldgVW7DXb8Ba8z3uhg2wbRsyFv//27v32LjKO43j33OZmy+JEydx4iQNlJIQAgQIBbFUpbvarlS65dYKSCW6rFqVqqy67IpKmwpFlUCo3faPrlpRJBrCAmJXbINKViqEViuhFrrNFkgIIWnSEhIHHGMwjh3bcznnffeP4xnPjG/j+8z4+UhR7Jk5x2OkzMN7+b0/vFwWNwjxHIvvWtzhwZ7vWhzXwXXD0lPc8lOsLsDwNKpjRw7odomK930PEjFsqglaVmBb2zEta6GlDZpWQqolCk8RKeFkB/F7O0h0Hyf13kHiZ47i953BSQ/i5IJos9BwWDp25DhGa4dHnA6Ajw0drHVwHQfjxgkaWwlWncfghVeSPv9ScqvXQypfyrKwZSmDgznuuONpvvvda7j99tk7+1qBWQemG5i7dx/hhz88wO7dt9LaOnq9LswFhL0fken+gMy7p8mdPk3Q1QX9/TiZDG4uh2MMnrWFQaGf/9qx0VKi5+A6Ab7nRUeNuS5O8UgRMzKijHlRMCaT2KYl2NaVsKIds3IttKyCJSsgpdGiyIyEIe5AN4n3/0zi3TdInjpEvOc07mAfTjYT1Xza6MAEx1iwBoeoVhTjgONhTf57wLqEyUZyTa2k119E5qIryH38YsLW1dE+gQWarj18uIt77nmOV165jQ0blszKPa21uK5bK82jQYE5WhiG9Pf3TykwX3jhJN/85kv87Ge3sGFDC0E6Q7anh0z3+wx0dJB+9z3CDz/ADAwOh2MW11pcx8Ez0cjRCw2u6+BZGz2Hg4/FNRbXc/Cd6PWuY/G84UPiPHB9P5pGTSWwzc3YlW2wuh2zah0sXwHLWmFK291FZNqsxR3oJfbBOyQ63iJ56gDxrhO4A7146Sw2DAEXTLTLNhqREq11GjAm6kdqHI9oj5FPkGwgt2IN2QsvJ3fxFYRrPw5LW0aOIJmnspTHHvsDv//9CX71q1tm5WhPYwy+7xOLTT4bVyUUmOWMMfT19VUcmP/7m3e4/cv/w85/2MTHEmcZOn2abM9H2MFByGZwwnDkjO/ocEn80EQhCbjG4Dru8MDQDu/DsdEBA6ZotOk5uPEYbkMCp7EZr20Vztp27Op2WNUGS5dDg4JRpNo4g/3EPjhJ4tRbJN4+SPLM2/i9H0I2M7zOGa1vWmOxBqyJjqcPLRjrYnCi5t7GIfDjhM0tBOsvINh8BWbzZdjlbTBcyziXZSlhaPj615/lc59bz44dM9/9r8CsA9Zazp49O2FgmiCg6+BBfvv0Pv75ySXcdkkfly0/F4WjjRb+PQcca6NAxMGzJnrOdUcC03VxwzAaadpo1Oi5Lp7n4ybi+KkUXmsr3tq1+OvW47atguXLcZuaqqJJs4hMnTvUj991isQ7h0kdf51Y5wn8c2chPYQxDta6WAthaDHWDo86IbAOxhmu/XSiUahJpQhaV2M3bcFu2YZdez5Oy/I5m649c6afO+74D37xi7/lk5+cWUcdBWYdyAdm/iDysZw5cIDXfv48/7i7ib+8IMu1K7pxrI3+GIPnRA2mCiNEp2gDq8PwOqWD44Afj+Mmk8RbWoivXk1i3Tpiq1fjrViB19KCM63jt0SkVjh9vcTfP0Xs7UOk/vgaXudp6D+LyQbR9KyNNg2FxgzP4LoYxxI6HsZxMO7wNK4fI2xphfXnw8VXwgWb8drW4cVjzGZw7tt3nIcffpn9+++gqWn664/GGOLx+KwdQzoPFJhj6e3tHTcwrTEc2vPffHXne3xipcsXNvaR6fkwWsDOjxbzgcnwBlULru8Ri8eJpVLEV6ygYe1akmvXEW9bRWz5cryGBhb6dA8RWWDW4vZ+iH/6T8SOHsT/0xu473fCYJogyA03rXMJneHRpxNN30YHE7lYx8U4TvR3UzN25Rq4eBv+psuIfezjuMkUs/E5s3Pnr2hqsjz66F9P+x4KzDpx9uzZcfu05XIhN//N0zg5w99dF5Dr7SXz0UfYbDZaa3RdPN8nlkiSaGok2dZG47p1NKxpJ7FyJfGWFtzamYIQkYWUy+F80IX/57dwj7yOd/I4tqcbE+YIrQUnGoGG1gy35XYIcbA+GMfHYqPusA0NmJZWvAsvJXbxlcTPuxB3ybJpH6wyOJjlttue5qGHruWLX7xwWvdQYNaJvr4+IKp52r9/P1deeSWu62KM4YknjnLvvS+zvi1GwsnhmSweIcmET0NzisaWZpqWNpNa2khDcyPJVIxEwh/+4xW+TiY94nGfZHLkuWTSLzwWi40/JSwii9RAP05nB87RA3DkIPbMaRjsx4Rm5KwEhs/8cizGAp5P6DiEjhM9H49jm5fhnbeJ5KVXET9/E/7ylThTDK5Dh87wrW/t5Xe/u53165un/KtYa4nH47XSCxMUmGPLN5EGeOihh3j++ee5++67uf3227HWpaNjkHM9ffR29TDYP4RNLcF4STJZSzodMDQUMDQUkk4HpNMBg4Ph8GMBmUxYeH5oKHo+/1j0+ui6MLTDAesVQrU4XEe+j4K3PJDLw7g4qEcC3C+5fzzuKaRFakUYQk839sQfMW+9in3nj/BRDyYzRIiLdfMvM9FBRJ6PtURTtp5L6ESbiVi6nNi680hechXJT2zBX7kGt8LwfPTR/bz++ileeOHmKZea1FjzaFBgjq2vr49cLlc4jqq3t5cf//jH/PrXv+b73/8+n/70p+f8PYShIZ0uDdLo69HhG4V0WPK6wcGg8Pr888XfF1+TyUR/Z7Mh8XgUoNGf0lFwPmyjgC4OZ49EIlYSvuXBnr/vSHDnr/OLashEZLrM4AD2vZPYowcJjr6Off8M4bmz0YjTdQlxwIbR6NNA6DlYz8c6DiY0WN/HbWzGa/8Yqc1X0Lj5CmIr10y4hBSGhq99bQ833XQe3/72tqm9XwVmfQiCgHQ6XdIuy3Ecuru7SafTrF+/fqHf4pwwxpaMcotHwcXhXR7O0Si6OMzDstH0yHWZTPH9R0K6eLSbHzmPNZU9EtCjg7d4JF7+WOkoWyEt9c0GAeb9dwnePkburT8QdJzAnO3BYgldFxuEWMdG4Tk8bWutIfA8rBP1znUam/BXtdOw5SoaNl5Kom0d3hin8nR29rF9+3+yd+8X2LatreL3WGPNo0GBOT5rLWEYks1mCcOwrChYZou1tmikHJaNikePkvOPZzJhYRRdPnrOP5bJlF8/MpqOxdxRIR2NpscO4/xrR6a/xw/j8oDPj9TnqhGvyESstZjeD8mdPkHu0B/InDhK2NNNGGSwOITWYB0v+mC3htBarOtiY3GsHW5p1thMfPVaGjZextKLt5FYvRY3NhKeL7xwjEce+R37999BY2NlmxoVmHVIwVl/rLVks2ZUmI41ei4fYReHc/nouXzUXTz9nU6HuK5TCOTiYB0rePNfFwf0WOvX5cFc/tpYrGZ2IMo8MQPnCDpPkT56gKEjB8n1nCEcGMR6LtZzMblctDnIWmwsgTUhUavPGNZ1IJEi3tbOkouuZMnFl5Nc2Y4bj3P//ftYtszjkUf+atL3kO/pq8CsUwpOmQlrLblc6br0eKPn8pFy8fT4WJvHSqfRR77OZMLhD6XSoJ14ynvsoC5evy4N/dL16WQyGknr30VtMNkMuc4Oho69ydCR18i8/x5maCCaujXDB8X7HkE2S4jF9X1sLIkxOXBcnEQD8ba1LNmyDX/dRr5672/41x9ez003XTDhz63BXpigwJw6BafUkiAw44bs6BH26Knt4rXp0lF16Yi6eG06v8N7rHAuDtvS6W6fWGzsKfHxNpKVTpVrh/dM2SAg98EZBo+/ycDhV8l0dpAb6McAeC5hEGDDAOvFsJ6PyaVxEsnokAQDblMTvXYJj70Y8Mxv/4nzNq0Z/2cpMBcXBafI2MLQlJRVTbYJrHxzWfHa9Ojp8ZFrRqa7A3I5UxbIE013jw7rsdeiRz9WGuD1G9ImDAk+7GLw7aP0v/kqQ50nyZ07G50uFIQQi0Wj0EwG63q4DU0YkyPevJwDJw1/7mvm31/6FxJNjWPevwZ7YYICc+bGCs4a2iYtUhfyO7wnmtouD+fSzWPlG89GX5cP53zwF+/wTia9USFbPAou3gk+uk567DAebyPZfO/wtmFIrqebc28f5ezhVxl67x2y/WexRJuDwmwGPA+/eRnx5qUkVraz6+en+IsvXs99D9ww9j0VmIubglNkcTFmZId3+eh5srrpwcHRa9Njr2cHJbvIM5loh3fpYSbjjYJLDzWJx2Njrl+PXzs9MlLPH0pgrSXb0825t4/R++Z+Bt49SW6wHxMExJYuJdW2Hq+xiaH4Snb82wn+a9/fc/nlK0f9t6vB5tGgwJx9Ck4RmSvFO7wnm9ou38FdvnFsvHrrsdamPc8pCeNk0icRd2n2BlgedLI8fZKUOYttWIZZsganaQX/17mcdGB57bUvj9qtXYOtvWCcwFQ/qRlwHAff9/E8jyAIFJwiMmuiaczoyMyWlvmZzszv8C6vby5ZYx7I0N/VxWDPR+SMTzaxjA2Bh+NALpfDdaGjo4NnnnmG6667jmuvvXZe3vt8qJrA3LFjB21tbdx7770L/VZKXH311ezevZstW7aM+xrHcYjFYvi+r+AUkZrlOA7xeLQTeenSiV45upSkq6uLxx77GXv27CGVSvGlL32Jiy66aM7e60Koik/z7u5unnjiCe6++24AfvKTn3DVVVeRSCS46667Kr5PT08Pt9xyC42NjWzYsIGnn356xtfdd9997Ny5s6L75IOzoaGhUKQbhmHhcHcRkXr1yCOPYK3lqaeeYu/evdx55500NzcXzuquB1WxhvmDH/yAY8eO8eijjwLw7LPP4rou+/btY2hoiMcff7yi+2zfvh1jDLt27eLAgQN8/vOf55VXXplwdDjZdel0mvb2dg4fPsyaNePXGo3FWlsYcRpjxm1ULSJSD/J5kj/dx/M8fN/H9/1am20b84O6Kn6D559/nuuvv77w/a233srNN99Ma2trxfcYGBhgz549PPDAAzQ1NfGpT32KG2+8kSeffHJG1yWTSbZt28aLL7445d+rfMQJ0QL4JP+TIiJSU/IbIPOzafF4nMbGRhoaGmqtD+aEquK3OHToEJs2bZrRPY4dO4bneWzcuLHw2NatWzl8+PCMr9u8eTMHDx6c9nsrDs58LZKCU0Rq2Vgh2dDQUHchWawqNv309vbS3Dz1Lt7Fzp07x9KyVeqlS5fS398/4+uam5vp7Oyc0fuD8TcHaapWRGpBfqo1fxhBPB4vTLcuhs+wqgjMZcuWTRpsk2lqaqKvr6/ksb6+vkmDuJLr+vv7aWlpmdH7K6bgFJFakQ/IvPyapOfV73GB46mKMfNll13GsWPHZnSPjRs3EgQBx48fLzx28ODBSTf8VHLdkSNH2Lp164ze31jGWuMMw1BTtSKyoKy1GGMKU66e55FMJmlsbCSZTOL7/qILS6iSwLzhhht46aWXCt8HQUA6nSYMQ8IwJJ1OEwRB4fm77rprVLlJY2Mjt956Kzt37mRgYICXX36Z5557jjvvvHNG12UyGV599VU++9nPzv4vPkzBKSILrTwkXdclmUzS1NREKpVatCFZrCoC8ytf+Qq//OUvGRoaAuDBBx8klUrxve99j6eeeopUKsWDDz5YeH1HRwfXXXfdqPs8/PDDDA0NsWrVKrZv385Pf/rTkpHidK7bu3cvn/nMZ2hvb5/tX3sUBaeIzKf8dKsxphCSiUSisMM1Fost+pAsVhV1mADf+c53WLVq1aQn/WSzWbZu3cobb7wxpbMJp3vdNddcw65du7jkkksqvma2RMdU5cjlcqrjFJFZkw9JANd1C3sq6nFn6zTp8PVapeAUkZnKh2S+l2/xgQL6PBlFgVnrFJwiMhXlZSDFI0l9dkyoek/6kcrk656KD0DQGqeIFCtek7TW4vs+qVSKxsZGEonEvJSD7Nixgx/96Edz+jOm4+qrr570MJuJKDBrkIJTRIpVUxlIeTON6TbFmIsmHFNppjGWqji4QKYnH5yxWKwwVasDEEQWh+LpVgDP8wpTrgv57//xxx/nhhtuIJVKAXDPPfcQj8fp6uoqNLfYunXrpDXy7e3t3H///YUmHJWa6OfdeOONfOMb36Czs3PKzTRAI8y6oBGnyOJQC2Ugxc00ptsUA+amCcdMmmmAArOulAdn/nBkBadIbaulbiDFzTSm2xRjuua6mYamZOtQ+VSt+nGK1J7yMpBaOei8uJnGdJtiTNdcN9NQYNYxBadIbamHbiDFzTSm2xRjuua6mUb1jONlzuT/4eW3lWuqVqR6VEMZyGwqbqYx3aYY0zXXzTQUmIuIglOkOlRTGchsK26mMd2mGDA3TThm2kxDgbkIKThF5t9i6QZS3kxjuk0x5qIJx0ybaehoPCkcuZfNZrHW1tR6iUg1y3++5tclPc8rOcO1XtVBMw2dJSsTKw5OoLA7T0SmRt1Aap4CUyqj4BSZOnUDqSsKTJkaBafIxNQNpG4pMGV6FJwiI4rPbwXwfZ9YLKaQrC8KTJkZBacsVuUHnedDstZqJKViCkyZHdZastksuVwOUHBKfarWbiAyLxSYMrsUnFJvFmsZiIyiwJS5oeCUWqcyECmjwJS5peCUWqIyEJmAAlPmhzGGXC6n4JSqozIQqZACU+aXglOqgcpAZBoUmLIwioNTZ9XKfFAZiMyQAlMWVj4483WcCk6ZTSoDkVmkwJTqoOCU2aIyEJkjCkypLgpOmS6VgcgcU2BKdVJwSiVUBiLzSIEp1U3BKeVUBiILRIEptUHBubipDESqgAJTaouCc/FQGYhUGQWm1CYFZ31SGYhUMQWm1DZjTMlZtQrO2qMyEKkRCkypD/ngDIJAJwfVCJWBSI1RYEp9UXBWN5WBSA1TYEp9UnBWD5WBSJ1QYEp9U3AuDJWBSB1SYMrioOCceyoDkTqnwJTFRcE5u1QGIouIAlMWp+LgBDWyngqVgcgipcCUxU3BWTmVgcgip8AUgdIDEPKhqeBUGYhIEQWmSDEFp8pARMahwBQZy2ILTpWBiExKgSkykTAMC2uc9RacKgMRmRIFpkgl6iU4VQYiMm0KTJGpqMXgVBmIyKxQYIpMRy0Ep8pARGaVAlNkJqotOFUGIjJnFJgis2Ehg1NlICLzQoEpMpvmKzhVBiIy7xSYInNhLoJTZSAiC0qBKTKXZhqcKgMRqRoKTJG5lt+IU2lwqgxEpCopMEXmSz44M5kMYRiOCk6VgYhUNQWmyHwrD05AZSAi1U+BKbJQ8sEZBIFCUqT6KTBFREQqMGZgasFERESkAgpMERGRCigwRUREKqDAFBERqYACU0REpAIKTBERkQooMEVERCqgwBQREamAAlNERKQCCkwREZEKKDBFREQqoMAUERGpgAJTRESkAgpMERGRCigwRUREKqDAFBERqYA/yfNqCS8iIoJGmCIiIhVRYIqIiFRAgSkiIlIBBaaIiEgFFJgiIiIVUGCKiIhU4P8B2PdiwZEU0/AAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/kolmogorov_fwd_7_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "Q = np.array(Q)\n", + "ψ_00 = np.array((0.01, 0.01, 0.99))\n", + "ψ_01 = np.array((0.01, 0.99, 0.01))\n", + "ψ_02 = np.array((0.99, 0.01, 0.01))\n", + "\n", + "ax = unit_simplex(angle=50) \n", + "\n", + "def flow_plot(ψ, h=0.001, n=400, angle=50):\n", + " colors = cm.jet_r(np.linspace(0.0, 1, n))\n", + "\n", + " x_vals, y_vals, z_vals = [], [], []\n", + " for t in range(n):\n", + " x_vals.append(ψ[0])\n", + " y_vals.append(ψ[1])\n", + " z_vals.append(ψ[2])\n", + " ψ = ψ @ expm(h * Q)\n", + "\n", + " ax.scatter(x_vals, y_vals, z_vals, c=colors, s=20, alpha=0.2, depthshade=False)\n", + "\n", + "flow_plot(ψ_00)\n", + "flow_plot(ψ_01)\n", + "flow_plot(ψ_02)\n", + "\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "(Distributions cool over time, so initial conditions are hot colors.)\n", + "\n", + "### Forwards vs Backwards Equations\n", + "\n", + "As the above discussion shows, we can take the Kolmogorov forward equation\n", + "$P_t' = P_t Q$ and premultiply by any distribution $\\psi_0$ to get the\n", + "distribution ODE $\\psi'_t = \\psi_t Q$.\n", + "\n", + "In this sense, we can understand the Kolmogorov forward equation as pushing\n", + "distributions forward in time.\n", + "\n", + "\n", + "Analogously, we can take the Kolmogorov backward equation\n", + "$P_t' = Q P_t$ and postmultiply by any vector $h$ to get \n", + "\n", + "$$\n", + " (P_t h)' = Q P_t h\n", + "$$\n", + "\n", + "Recalling that $(P_t h)(x) = \\EE [ h(X_t) \\,|\\, X_0 = x]$, this vector\n", + "ODE tells us how expectations evolve, conditioning backward to time zero.\n", + "\n", + "Both the forward and the backward equations uniquely pin down the same solution $P_t = e^{tQ}$ when combined with the initial condition $P_0 = I$.\n", + "\n", + "\n", + "### Matrix- vs Vector-Valued ODEs\n", + "\n", + "The ODE $\\psi'_t = \\psi_t Q$ is sometimes called the\n", + "**Fokker--Planck equation** (although this terminology is most commonly used\n", + "in the context of diffusions).\n", + "\n", + "It is a vector-valued ODE that describes the evolution of a particular\n", + "distribution path.\n", + "\n", + "By comparison, the Kolmogorov forward equation is (like the backward equation)\n", + "a differential equation in matrices.\n", + "\n", + "(And matrices are really maps, that send vectors into vectors.)\n", + "\n", + "Operating at this level is less intuitive and more abstract than working with the\n", + "Fokker--Planck equation.\n", + "\n", + "But, in the end, the object that we want to describe is a the Markov\n", + "semigroup.\n", + "\n", + "The Kolmogorov forward and backward equations are the ODEs that define\n", + "this fundamental object.\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "### Preserving Distributions\n", + "\n", + "\n", + "In the simulation above, $Q$ was chosen with some care, so that the flow\n", + "remains in $\\dD$.\n", + "\n", + "What are the exact properties we require on $Q$ such that $\\psi_t$ is always\n", + "in $\\dD$?\n", + "\n", + "This is an important question, because we are setting up an exact\n", + "correspondence between linear ODEs that evolve in $\\dD$ and continuous\n", + "time Markov chains.\n", + "\n", + "Recall that the linear update rule $\\psi \\mapsto \\psi P$ is invariant on\n", + "$\\dD$ if and only if $P$ is a Markov matrix.\n", + "\n", + "So now we can rephrase our key question regarding invariance on $\\dD$:\n", + "\n", + "What properties do we need to impose on $Q$ so that $P_t = e^{tQ}$ is a Markov matrix\n", + "for all $t$?\n", + "\n", + "A square matrix $Q$ is called an **intensity matrix** if $Q$ has zero row\n", + "sums and $Q(x, y) \\geq 0$ whenever $x \\not= y$.\n", + "\n", + "```{proof:theorem}\n", + ":label: intvsmk\n", + "\n", + "If $Q$ is a matrix on $S$ and $P_t := e^{tQ}$, then the following statements\n", + "are equivalent:\n", + "\n", + "1. $P_t$ is a Markov matrix for all $t$.\n", + "1. $Q$ is an intensity matrix.\n", + "```\n", + "\n", + "The proof is related to that of {proof:ref}`jctosg` and is found as \n", + "a solved exercise below.\n", + "\n", + "```{proof:corollary}\n", + ":label: intvsmk_c\n", + "\n", + "If $Q$ is an intensity matrix on finite $S$ and $P_t = e^{tQ}$ for all $t \\geq 0$,\n", + "then $(P_t)$ is a Markov semigroup.\n", + "```\n", + "\n", + "We call $(P_t)$ the Markov semigroup **generated** by $Q$.\n", + "\n", + "Later we will see that this result extends to the case $|S| = \\infty$ under\n", + "some mild restrictions on $Q$.\n", + "\n", + "\n", + "\n", + "## Jump Chains\n", + "\n", + "Let's return to the chain $(X_t)$ created from jump chain pair $(\\lambda, K)$ in\n", + "{proof:ref}`ejc_algo`.\n", + "\n", + "We found that the semigroup is given by \n", + "\n", + "$$ \n", + " P_t = e^{tQ}\n", + " \\quad \\text{where} \\quad\n", + " Q(x, y) := \\lambda(x) (K(x, y) - I(x, y))\n", + "$$\n", + "\n", + "Using the fact that $K$ is a Markov matrix and the jump rate function\n", + "$\\lambda$ is nonnegative, you can easily check that $Q$ satisfies the\n", + "definition of an intensity matrix.\n", + "\n", + "Hence $(P_t)$, the Markov semigroup for the jump chain $(X_t)$, is the\n", + "semigroup generated by the intensity matrix $Q(x, y) = \\lambda(x) (K(x, y) - I(x, y))$.\n", + "\n", + "We can differentiate $P_t = e^{tQ}$ to obtain the Kolmogorov forward equation\n", + "$P_t' = P_t Q$.\n", + "\n", + "We can then premultiply by $\\psi_0 \\in \\dD$ to get $\\psi_t' = \\psi_t\n", + "Q$, which is the Fokker--Planck equation.\n", + "\n", + "More explicitly, for given $y \\in S$,\n", + "\n", + "$$\n", + " \\psi_t'(y) \n", + " = \\sum_{x \\not= y} \\psi_t(x) \\lambda(x) K(x, y) - \\psi_t(y) \\lambda(y) \n", + "$$\n", + "\n", + "The rate of probability flow into $y$ is equal to the inflow from other states\n", + "minus the outflow.\n", + "\n", + "\n", + "\n", + "## Summary\n", + "\n", + "We have seen that any intensity matrix $Q$ on $S$ defines a Markov semigroup via $P_t = e^{tQ}$.\n", + "\n", + "Henceforth, we will say that $(X_t)$ is **a Markov chain with intensity matrix** $Q$ if \n", + "$(X_t)$ is a Markov chain with Markov semigroup $(e^{tQ})$.\n", + "\n", + "While our discussion has been in the context of a finite state space, later we\n", + "will see that these ideas carry over to an infinite state setting under mild\n", + "restrictions.\n", + "\n", + "We have also hinted at the fact that *every* continuous time Markov chain \n", + "is a Markov chain with intensity matrix $Q$ for some suitably chosen $Q$.\n", + "\n", + "Later we will prove this to be universally true when $S$ is finite and true\n", + "under mild conditions when $S$ is countably infinite.\n", + "\n", + "Intensity matrices are important because \n", + "\n", + "1. they are the natural infinitesimal descriptions of Markov semigroups,\n", + "2. they are often easy to write down in applications and\n", + "3. they provide an intuitive description of dynamics.\n", + "\n", + "\n", + "Later, we will see that, for a given intensity matrix $Q$, the elements are\n", + " understood as follows:\n", + "\n", + "* when $x \\not= y$, the value $Q(x, y)$ is the \"rate of leaving $x$ for $y$\" and\n", + "* $-Q(x, x) \\geq 0$ is the \"rate of leaving $x$\" .\n", + "\n", + "\n", + "\n", + "\n", + "## Exercises\n", + "\n", + "### Exercise 1\n", + "\n", + "Let $(P_t)$ be a Markov semigroup such that $t \\mapsto P_t(x, y)$ is\n", + "differentiable at all $t \\geq 0$ and $(x, y) \\in S \\times S$.\n", + "\n", + "(The derivative at $t=0$ is the usual right derivative.)\n", + "\n", + "Define (pointwise, at each $(x,y)$),\n", + "\n", + "$$ \n", + " Q := P'_0 = \\lim_{h \\downarrow 0} \\frac{P_h - I}{h}\n", + "$$ (genfl)\n", + "\n", + "Assuming that this limit exists, and hence $Q$ is well-defined, show that\n", + "\n", + "$$\n", + " P'_t = P_t Q\n", + " \\quad \\text{and} \\quad\n", + " P'_t = Q P_t \n", + "$$\n", + "\n", + "both hold. (These are the Kolmogorov forward and backward equations.)\n", + "\n", + "\n", + "### Exercise 2\n", + "\n", + "Recall {ref}`our model ` of jump chains with state-dependent jump\n", + "intensities given by rate function $x \\mapsto \\lambda(x)$.\n", + "\n", + "After a wait time with exponential rate $\\lambda(x) \\in (0, \\infty)$, the\n", + "state transitions from $x$ to $y$ with probability $K(x,y)$.\n", + "\n", + "We found that the associated semigroup $(P_t)$ satisfies the Kolmogorov\n", + "backward equation $P'_t = Q P_t$ with \n", + "\n", + "$$\n", + " Q(x, y) := \\lambda(x) (K(x, y) - I(x, y))\n", + "$$ (qeqagain)\n", + "\n", + "Show that $Q$ is an intensity matrix and that {eq}`genfl` holds.\n", + "\n", + "### Exercise 3 \n", + "\n", + "Prove {proof:ref}`intvsmk` by adapting the arguments in {proof:ref}`jctosg`.\n", + "(This is nontrivial but worth at least trying.)\n", + "\n", + "Hint: The constant $m$ in the proof can be set to $\\max_x |Q(x, x)|$.\n", + "\n", + "\n", + "\n", + "## Solutions\n", + "\n", + "### Solution to Exercise 1\n", + "\n", + "Let $(P_t)$ be a Markov semigroup and let $Q$ be as defined in the statement\n", + "of the exercise.\n", + "\n", + "Fix $t \\geq 0$ and $h > 0$. \n", + "\n", + "Combining the semigroup property and linearity with the restriction $P_0 = I$, we get\n", + "\n", + "$$\n", + " \\frac{P_{t+h} - P_t}{h} \n", + " = \\frac{P_t P_h - P_t}{h} \n", + " = \\frac{P_t (P_h - I)}{h} \n", + "$$\n", + "\n", + "Taking $h \\downarrow 0$ and using the definition of $Q$ gives $P_t' = P_t Q$,\n", + "which is the Kolmogorov forward equation.\n", + "\n", + "For the backward equation we observe that \n", + "\n", + "$$\n", + " \\frac{P_{t+h} - P_t}{h} \n", + " = \\frac{P_h P_t - P_t}{h} \n", + " = \\frac{(P_h - I) P_t}{h} \n", + "$$\n", + "\n", + "also holds. Taking $h \\downarrow 0$ gives the Kolmogorov backward equation.\n", + "\n", + "\n", + "\n", + "### Solution to Exercise 2\n", + "\n", + "Let $Q$ be as defined in {eq}`qeqagain`.\n", + "\n", + "We need to show that $Q$ is nonnegative off the diagonal and has zero row\n", + "sums.\n", + "\n", + "The first assertion is immediate from nonnegativity of $K$ and $\\lambda$.\n", + "\n", + "For the second, we use the fact that $K$ is a Markov matrix, so that, with $1$\n", + "as a column vector of ones,\n", + "\n", + "$$\n", + " Q 1 \n", + " = \\lambda (K 1 - 1)\n", + " = \\lambda (1 - 1)\n", + " = 0\n", + "$$\n", + "\n", + "### Solution to Exercise 3\n", + "\n", + "\n", + "Suppose that $Q$ is an intensity matrix, fix $t \\geq 0$ and set $P_t = e^{tQ}$.\n", + "\n", + "The proof from {proof:ref}`jctosg` that $P_t$ has unit row sums applies\n", + "directly to the current case.\n", + "\n", + "The proof of nonnegativity of $P_t$ can be applied after some\n", + "modifications.\n", + "\n", + "To this end, set $m := \\max_x |Q(x,x)|$ and $\\hat P := I + Q / m$.\n", + "\n", + "You can check that $\\hat P$ is a Markov matrix and that $Q = m( \\hat P - I)$.\n", + "\n", + "The rest of the proof of nonnegativity of $P_t$ is unchanged and we will not repeat it.\n", + "\n", + "We conclude that $P_t$ is a Markov matrix.\n", + "\n", + "Regarding the converse implication, suppose that $P_t = e^{tQ}$ is a Markov\n", + "matrix for all $t$ and let $1$ be a column vector of ones.\n", + "\n", + "Because $P_t$ has unit row sums and differentiation is linear, \n", + "we can employ the Kolmogorov backward equation to obtain\n", + "\n", + "$$\n", + " Q 1\n", + " = Q P_t 1\n", + " = \\left( \\frac{d}{d t} P_t \\right) 1\n", + " = \\frac{d}{d t} (P_t 1)\n", + " = \\frac{d}{d t} 1\n", + " = 0\n", + "$$\n", + "\n", + "Hence $Q$ has zero row sums.\n", + "\n", + "We can use the definition of the matrix exponential to obtain,\n", + " for any $x, y$ and $t \\geq 0$,\n", + "\n", + "$$\n", + " P_t(x, y) = \\mathbb 1\\{x = y\\} + t Q(x, y) + o(t)\n", + "$$ (otp)\n", + "\n", + "From this equality and the assumption that $P_t$ is a Markov matrix for all\n", + "$t$, we see that the off diagonal elements of $Q$ must be\n", + "nonnegative.\n", + "\n", + "Hence $Q$ is a intensity matrix." + ] + } + ], + "metadata": { + "jupytext": { + "formats": "ipynb,md:myst", + "text_representation": { + "extension": ".md", + "format_name": "myst", + "format_version": "0.9", + "jupytext_version": "1.5.0" + } + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.7" + }, + "source_map": [ + 13, + 45, + 56, + 93, + 99, + 163, + 271, + 277, + 304 + ] + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file diff --git a/_sources/kolmogorov_fwd.md b/_sources/kolmogorov_fwd.md new file mode 100644 index 0000000..e02ceff --- /dev/null +++ b/_sources/kolmogorov_fwd.md @@ -0,0 +1,632 @@ +--- +jupytext: + formats: ipynb,md:myst + text_representation: + extension: .md + format_name: myst + format_version: '0.9' + jupytext_version: 1.5.0 +kernelspec: + display_name: Python 3 + language: python + name: python3 +--- + +# The Kolmogorov Forward Equation + + + +## Overview + +In this lecture we approach continuous time Markov chains from a more +analytical perspective. + +The emphasis will be on describing distribution flows through vector-valued +differential equations and their solutions. + +These distribution flows show how the time $t$ distribution associated with a +given Markov chain $(X_t)$ changes over time. + +Distribution flows will be identified with initial value problems generated by autonomous linear ordinary differential equations (ODEs) in vector space. + +We will see that the solutions of these flows are described by Markov semigroups. + +This leads us back to the theory we have already constructed -- some care will +be taken to clarify all the connections. + +In order to avoid being distracted by technicalities, we continue to defer our +treatment of infinite state spaces, assuming throughout this lecture that $|S| += n$. + +As before, $\dD$ is the set of all distributions on $S$. + +We will use the following imports + +```{code-cell} ipython3 +import numpy as np +import scipy as sp +import matplotlib.pyplot as plt +import quantecon as qe +from numba import njit +from scipy.linalg import expm + +from matplotlib import cm +from mpl_toolkits.mplot3d import Axes3D +from mpl_toolkits.mplot3d.art3d import Poly3DCollection +``` + +## From Difference Equations to ODEs + +{ref}`Previously ` we generated this figure, which shows how distributions evolve over time for the inventory model under a certain parameterization: + +```{glue:figure} flow_fig +Probability flows for the inventory model. +``` + +(Hot colors indicate early dates and cool colors denote later dates.) + +We also learned how this flow is related to +the Kolmogorov backward equation, which is an ODE. + +In this section we examine distribution flows and their connection to +ODEs and continuous time Markov chains more systematically. + + +### Review of the Discrete Time Case + +Let $(X_t)$ be a discrete time Markov chain with Markov matrix $P$. + +{ref}`Recall that `, in the discrete time case, the distribution $\psi_t$ of $X_t$ updates according to + +$$ + \psi_{t+1} = \psi_t P, + \qquad \psi_0 \text{ a given element of } \dD, +$$ + +where distributions are understood as row vectors. + +Here's a visualization for the case $|S|=3$, so that $\dD$ is the unit +simplex in $\RR^3$. + +The initial condition is `` (0, 0, 1)`` and the Markov matrix is + +```{code-cell} ipython3 +P = ((0.9, 0.1, 0.0), + (0.4, 0.4, 0.2), + (0.1, 0.1, 0.8)) +``` + +```{code-cell} ipython3 +:tags: [hide-input] + +def unit_simplex(angle): + + fig = plt.figure(figsize=(8, 6)) + ax = fig.add_subplot(111, projection='3d') + + vtx = [[0, 0, 1], + [0, 1, 0], + [1, 0, 0]] + + tri = Poly3DCollection([vtx], color='darkblue', alpha=0.3) + tri.set_facecolor([0.5, 0.5, 1]) + ax.add_collection3d(tri) + + ax.set(xlim=(0, 1), ylim=(0, 1), zlim=(0, 1), + xticks=(1,), yticks=(1,), zticks=(1,)) + + ax.set_xticklabels(['$(1, 0, 0)$'], fontsize=12) + ax.set_yticklabels(['$(0, 1, 0)$'], fontsize=12) + ax.set_zticklabels(['$(0, 0, 1)$'], fontsize=12) + + ax.xaxis.majorTicks[0].set_pad(15) + ax.yaxis.majorTicks[0].set_pad(15) + ax.zaxis.majorTicks[0].set_pad(35) + + ax.view_init(30, angle) + + # Move axis to origin + ax.xaxis._axinfo['juggled'] = (0, 0, 0) + ax.yaxis._axinfo['juggled'] = (1, 1, 1) + ax.zaxis._axinfo['juggled'] = (2, 2, 0) + + ax.grid(False) + + return ax + + +def convergence_plot(ψ, n=14, angle=50): + + ax = unit_simplex(angle) + + P = ((0.9, 0.1, 0.0), + (0.4, 0.4, 0.2), + (0.1, 0.1, 0.8)) + + P = np.array(P) + colors = cm.jet_r(np.linspace(0.0, 1, n)) + + x_vals, y_vals, z_vals = [], [], [] + for t in range(n): + x_vals.append(ψ[0]) + y_vals.append(ψ[1]) + z_vals.append(ψ[2]) + ψ = ψ @ P + + ax.scatter(x_vals, y_vals, z_vals, c=colors, s=50, alpha=0.7, depthshade=False) + + return ψ + +ψ = convergence_plot((0, 0, 1)) + +plt.show() +``` + +There's a sense in which a discrete time Markov chain "is" a homogeneous +linear difference equation in distribution space. + +To clarify this, suppose we +take $G$ to be a linear map from $\dD$ to itself and +write down the difference equation + +$$ + \psi_{t+1} = G(\psi_t) + \quad \text{with } \psi_0 \in \dD \text{ given}. +$$ (gdiff2) + +Because $G$ is a linear map from a finite dimensional space to itself, it can +be represented by a matrix. + +Moreover, a matrix $P$ is a Markov matrix if and only if $\psi \mapsto +\psi P$ sends $\dD$ into itself (check it if you haven't already). + +So, under the stated conditions, our difference equation {eq}`gdiff2` uniquely +identifies a Markov matrix, along with an initial condition $\psi_0$. + +Together, these objects identify the joint distribution of a discrete time Markov chain, as {ref}`previously described `. + + +### Shifting to Continuous Time + +We have just argued that a discrete time Markov chain can be identified with a +linear difference equation evolving in $\dD$. + +This strongly suggests that a continuous time Markov chain can be identified +with a linear ODE evolving in $\dD$. + +This intuition is correct and important. + +The rest of the lecture maps out the main ideas. + + + +## ODEs in Distribution Space + +Consider linear differential equation given by + +$$ + \psi_t' = \psi_t Q, + \qquad \psi_0 \text{ a given element of } \dD, +$$ (ode_mc) + +where + +* $Q$ is an $n \times n$ matrix, +* distributions are again understood as row vectors, and +* derivatives are taken element by element, so that + +$$ + \psi_t' = + \begin{pmatrix} + \frac{d}{dt} \psi_t(x_1) & + \cdots & + \frac{d}{dt} \psi_t(x_n) + \end{pmatrix} +$$ + +### Solutions to Linear Vector ODEs + +Using the matrix exponential, the unique solution to the initial value problem +{eq}`ode_mc` is + +$$ + \psi_t = \psi_0 P_t + \quad \text{where } P_t := e^{tQ} +$$ (cmc_sol) + +To check that {eq}`cmc_sol` is a solution, we use {eq}`expoderiv` again to get + +$$ + \frac{d}{d t} P_t = Q e^{tQ} = e^{tQ} Q +$$ + +The first equality can be written as $P_t' = Q P_t$ and this is just +the {doc}`Kolmogorov backward equation `. + +The second equality can be written as + +$$ + P_t' = P_t Q +$$ + +and is called the **Kolmogorov forward equation**. + +Applying the Kolmogorov forward equation, we obtain + +$$ + \frac{d}{d t} \psi_t + = \frac{d}{d t} \psi_0 P_t + = \psi_0 \frac{d}{d t} P_t + = \psi_0 P_t Q + = \psi_t Q +$$ + +This confirms that {eq}`cmc_sol` solves {eq}`ode_mc`. + + +Here's an example of three distribution flows with dynamics generated by {eq}`ode_mc`, one starting from each vertex. + +The code uses {eq}`cmc_sol` with matrix $Q$ given by + +```{code-cell} ipython3 +Q = ((-3, 2, 1), + (3, -5, 2), + (4, 6, -10)) +``` + +```{code-cell} ipython3 +:tags: [hide-input] + +Q = np.array(Q) +ψ_00 = np.array((0.01, 0.01, 0.99)) +ψ_01 = np.array((0.01, 0.99, 0.01)) +ψ_02 = np.array((0.99, 0.01, 0.01)) + +ax = unit_simplex(angle=50) + +def flow_plot(ψ, h=0.001, n=400, angle=50): + colors = cm.jet_r(np.linspace(0.0, 1, n)) + + x_vals, y_vals, z_vals = [], [], [] + for t in range(n): + x_vals.append(ψ[0]) + y_vals.append(ψ[1]) + z_vals.append(ψ[2]) + ψ = ψ @ expm(h * Q) + + ax.scatter(x_vals, y_vals, z_vals, c=colors, s=20, alpha=0.2, depthshade=False) + +flow_plot(ψ_00) +flow_plot(ψ_01) +flow_plot(ψ_02) + +plt.show() +``` + +(Distributions cool over time, so initial conditions are hot colors.) + +### Forwards vs Backwards Equations + +As the above discussion shows, we can take the Kolmogorov forward equation +$P_t' = P_t Q$ and premultiply by any distribution $\psi_0$ to get the +distribution ODE $\psi'_t = \psi_t Q$. + +In this sense, we can understand the Kolmogorov forward equation as pushing +distributions forward in time. + + +Analogously, we can take the Kolmogorov backward equation +$P_t' = Q P_t$ and postmultiply by any vector $h$ to get + +$$ + (P_t h)' = Q P_t h +$$ + +Recalling that $(P_t h)(x) = \EE [ h(X_t) \,|\, X_0 = x]$, this vector +ODE tells us how expectations evolve, conditioning backward to time zero. + +Both the forward and the backward equations uniquely pin down the same solution $P_t = e^{tQ}$ when combined with the initial condition $P_0 = I$. + + +### Matrix- vs Vector-Valued ODEs + +The ODE $\psi'_t = \psi_t Q$ is sometimes called the +**Fokker--Planck equation** (although this terminology is most commonly used +in the context of diffusions). + +It is a vector-valued ODE that describes the evolution of a particular +distribution path. + +By comparison, the Kolmogorov forward equation is (like the backward equation) +a differential equation in matrices. + +(And matrices are really maps, that send vectors into vectors.) + +Operating at this level is less intuitive and more abstract than working with the +Fokker--Planck equation. + +But, in the end, the object that we want to describe is a the Markov +semigroup. + +The Kolmogorov forward and backward equations are the ODEs that define +this fundamental object. + + + + + +### Preserving Distributions + + +In the simulation above, $Q$ was chosen with some care, so that the flow +remains in $\dD$. + +What are the exact properties we require on $Q$ such that $\psi_t$ is always +in $\dD$? + +This is an important question, because we are setting up an exact +correspondence between linear ODEs that evolve in $\dD$ and continuous +time Markov chains. + +Recall that the linear update rule $\psi \mapsto \psi P$ is invariant on +$\dD$ if and only if $P$ is a Markov matrix. + +So now we can rephrase our key question regarding invariance on $\dD$: + +What properties do we need to impose on $Q$ so that $P_t = e^{tQ}$ is a Markov matrix +for all $t$? + +A square matrix $Q$ is called an **intensity matrix** if $Q$ has zero row +sums and $Q(x, y) \geq 0$ whenever $x \not= y$. + +```{proof:theorem} +:label: intvsmk + +If $Q$ is a matrix on $S$ and $P_t := e^{tQ}$, then the following statements +are equivalent: + +1. $P_t$ is a Markov matrix for all $t$. +1. $Q$ is an intensity matrix. +``` + +The proof is related to that of {proof:ref}`jctosg` and is found as +a solved exercise below. + +```{proof:corollary} +:label: intvsmk_c + +If $Q$ is an intensity matrix on finite $S$ and $P_t = e^{tQ}$ for all $t \geq 0$, +then $(P_t)$ is a Markov semigroup. +``` + +We call $(P_t)$ the Markov semigroup **generated** by $Q$. + +Later we will see that this result extends to the case $|S| = \infty$ under +some mild restrictions on $Q$. + + + +## Jump Chains + +Let's return to the chain $(X_t)$ created from jump chain pair $(\lambda, K)$ in +{proof:ref}`ejc_algo`. + +We found that the semigroup is given by + +$$ + P_t = e^{tQ} + \quad \text{where} \quad + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +$$ + +Using the fact that $K$ is a Markov matrix and the jump rate function +$\lambda$ is nonnegative, you can easily check that $Q$ satisfies the +definition of an intensity matrix. + +Hence $(P_t)$, the Markov semigroup for the jump chain $(X_t)$, is the +semigroup generated by the intensity matrix $Q(x, y) = \lambda(x) (K(x, y) - I(x, y))$. + +We can differentiate $P_t = e^{tQ}$ to obtain the Kolmogorov forward equation +$P_t' = P_t Q$. + +We can then premultiply by $\psi_0 \in \dD$ to get $\psi_t' = \psi_t +Q$, which is the Fokker--Planck equation. + +More explicitly, for given $y \in S$, + +$$ + \psi_t'(y) + = \sum_{x \not= y} \psi_t(x) \lambda(x) K(x, y) - \psi_t(y) \lambda(y) +$$ + +The rate of probability flow into $y$ is equal to the inflow from other states +minus the outflow. + + + +## Summary + +We have seen that any intensity matrix $Q$ on $S$ defines a Markov semigroup via $P_t = e^{tQ}$. + +Henceforth, we will say that $(X_t)$ is **a Markov chain with intensity matrix** $Q$ if +$(X_t)$ is a Markov chain with Markov semigroup $(e^{tQ})$. + +While our discussion has been in the context of a finite state space, later we +will see that these ideas carry over to an infinite state setting under mild +restrictions. + +We have also hinted at the fact that *every* continuous time Markov chain +is a Markov chain with intensity matrix $Q$ for some suitably chosen $Q$. + +Later we will prove this to be universally true when $S$ is finite and true +under mild conditions when $S$ is countably infinite. + +Intensity matrices are important because + +1. they are the natural infinitesimal descriptions of Markov semigroups, +2. they are often easy to write down in applications and +3. they provide an intuitive description of dynamics. + + +Later, we will see that, for a given intensity matrix $Q$, the elements are + understood as follows: + +* when $x \not= y$, the value $Q(x, y)$ is the "rate of leaving $x$ for $y$" and +* $-Q(x, x) \geq 0$ is the "rate of leaving $x$" . + + + + +## Exercises + +### Exercise 1 + +Let $(P_t)$ be a Markov semigroup such that $t \mapsto P_t(x, y)$ is +differentiable at all $t \geq 0$ and $(x, y) \in S \times S$. + +(The derivative at $t=0$ is the usual right derivative.) + +Define (pointwise, at each $(x,y)$), + +$$ + Q := P'_0 = \lim_{h \downarrow 0} \frac{P_h - I}{h} +$$ (genfl) + +Assuming that this limit exists, and hence $Q$ is well-defined, show that + +$$ + P'_t = P_t Q + \quad \text{and} \quad + P'_t = Q P_t +$$ + +both hold. (These are the Kolmogorov forward and backward equations.) + + +### Exercise 2 + +Recall {ref}`our model ` of jump chains with state-dependent jump +intensities given by rate function $x \mapsto \lambda(x)$. + +After a wait time with exponential rate $\lambda(x) \in (0, \infty)$, the +state transitions from $x$ to $y$ with probability $K(x,y)$. + +We found that the associated semigroup $(P_t)$ satisfies the Kolmogorov +backward equation $P'_t = Q P_t$ with + +$$ + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +$$ (qeqagain) + +Show that $Q$ is an intensity matrix and that {eq}`genfl` holds. + +### Exercise 3 + +Prove {proof:ref}`intvsmk` by adapting the arguments in {proof:ref}`jctosg`. +(This is nontrivial but worth at least trying.) + +Hint: The constant $m$ in the proof can be set to $\max_x |Q(x, x)|$. + + + +## Solutions + +### Solution to Exercise 1 + +Let $(P_t)$ be a Markov semigroup and let $Q$ be as defined in the statement +of the exercise. + +Fix $t \geq 0$ and $h > 0$. + +Combining the semigroup property and linearity with the restriction $P_0 = I$, we get + +$$ + \frac{P_{t+h} - P_t}{h} + = \frac{P_t P_h - P_t}{h} + = \frac{P_t (P_h - I)}{h} +$$ + +Taking $h \downarrow 0$ and using the definition of $Q$ gives $P_t' = P_t Q$, +which is the Kolmogorov forward equation. + +For the backward equation we observe that + +$$ + \frac{P_{t+h} - P_t}{h} + = \frac{P_h P_t - P_t}{h} + = \frac{(P_h - I) P_t}{h} +$$ + +also holds. Taking $h \downarrow 0$ gives the Kolmogorov backward equation. + + + +### Solution to Exercise 2 + +Let $Q$ be as defined in {eq}`qeqagain`. + +We need to show that $Q$ is nonnegative off the diagonal and has zero row +sums. + +The first assertion is immediate from nonnegativity of $K$ and $\lambda$. + +For the second, we use the fact that $K$ is a Markov matrix, so that, with $1$ +as a column vector of ones, + +$$ + Q 1 + = \lambda (K 1 - 1) + = \lambda (1 - 1) + = 0 +$$ + +### Solution to Exercise 3 + + +Suppose that $Q$ is an intensity matrix, fix $t \geq 0$ and set $P_t = e^{tQ}$. + +The proof from {proof:ref}`jctosg` that $P_t$ has unit row sums applies +directly to the current case. + +The proof of nonnegativity of $P_t$ can be applied after some +modifications. + +To this end, set $m := \max_x |Q(x,x)|$ and $\hat P := I + Q / m$. + +You can check that $\hat P$ is a Markov matrix and that $Q = m( \hat P - I)$. + +The rest of the proof of nonnegativity of $P_t$ is unchanged and we will not repeat it. + +We conclude that $P_t$ is a Markov matrix. + +Regarding the converse implication, suppose that $P_t = e^{tQ}$ is a Markov +matrix for all $t$ and let $1$ be a column vector of ones. + +Because $P_t$ has unit row sums and differentiation is linear, +we can employ the Kolmogorov backward equation to obtain + +$$ + Q 1 + = Q P_t 1 + = \left( \frac{d}{d t} P_t \right) 1 + = \frac{d}{d t} (P_t 1) + = \frac{d}{d t} 1 + = 0 +$$ + +Hence $Q$ has zero row sums. + +We can use the definition of the matrix exponential to obtain, + for any $x, y$ and $t \geq 0$, + +$$ + P_t(x, y) = \mathbb 1\{x = y\} + t Q(x, y) + o(t) +$$ (otp) + +From this equality and the assumption that $P_t$ is a Markov matrix for all +$t$, we see that the off diagonal elements of $Q$ must be +nonnegative. + +Hence $Q$ is a intensity matrix. + + diff --git a/_sources/markov_prop.ipynb b/_sources/markov_prop.ipynb new file mode 100644 index 0000000..5831042 --- /dev/null +++ b/_sources/markov_prop.ipynb @@ -0,0 +1,1207 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# The Markov Property \n", + "\n", + "## Overview\n", + "\n", + "\n", + "A continuous time stochastic process is said to have the Markov property if\n", + "that the past and future are independent given the current state.\n", + "\n", + "(A more formal definition is provided below.)\n", + "\n", + "As we will see, the Markov property imposes a large amount of structure on\n", + "continuous time processes.\n", + "\n", + "This structure leads to elegant and powerful results on\n", + "evolution and dynamics.\n", + "\n", + "At the same time, the Markov property is general enough to cover many applied\n", + "problems, as described in {doc}`the introduction `.\n", + "\n", + "\n", + "\n", + "### Setting\n", + "\n", + "In this lecture, the state space where dynamics\n", + "evolve will be a [countable set](https://en.wikipedia.org/wiki/Countable_set),\n", + "denoted henceforth by $S$, with typical elements $x, y$.\n", + "\n", + "(Note that \"countable\" is understood to include finite.)\n", + "\n", + "Regarding notation, in what follows, $\\sum_{x \\in S}$ is abbreviated to\n", + "$\\sum_x$, the supremum $\\sup_{x \\in S}$ is abbreviated to $\\sup_x$ and so on.\n", + "\n", + "A **distribution** on $S$ is a function $\\phi$ from $S$ to $\\RR_+$ with\n", + "$\\sum_x \\phi(x) = 1$.\n", + "\n", + "Let $\\dD$ denote the set of all distributions on $S$.\n", + "\n", + "To economize on terminology, we define a **matrix** $A$ on $S$ to be a map \n", + "from $S \\times S$ to $\\RR$.\n", + "\n", + "When $S$ is finite, this reduces to the usual notion of a matrix, and,\n", + "whenever you see expressions such as $A(x,y)$ below, you can\n", + "mentally identify them with more familiar matrix\n", + "notation, such as $A_{ij}$, if you wish.\n", + "\n", + "The product of two matrices $A$ and $B$ is defined by \n", + "\n", + "$$\n", + " (A B)(x, y) = \\sum_z A(x, z) B(z, y)\n", + " \\qquad ((x, y) \\in S \\times S)\n", + "$$ (kernprod)\n", + "\n", + "If $S$ is finite, then this is just ordinary matrix multiplication.\n", + "\n", + "In statements involving matrix algebra, we *always treat distributions as row\n", + "vectors*, so that, for $\\phi \\in \\dD$ and given matrix $A$,\n", + "\n", + "$$\n", + " (\\phi A)(y) = \\sum_x \\phi(x) A(x, y) \n", + "$$\n", + "\n", + "We will use the following imports" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import scipy as sp\n", + "import matplotlib.pyplot as plt\n", + "\n", + "import quantecon as qe\n", + "from numba import njit\n", + "\n", + "from scipy.linalg import expm\n", + "from scipy.stats import binom\n", + "\n", + "from matplotlib import cm\n", + "from mpl_toolkits.mplot3d import Axes3D" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Markov Processes\n", + "\n", + "We now introduce the definition of Markov processes, first reviewing the\n", + "discrete case and then shifting to continuous time.\n", + "\n", + "\n", + "\n", + "\n", + "(finstatediscretemc)=\n", + "### Discrete Time, Finite State \n", + "\n", + "The simplest Markov processes are those with a discrete time parameter and finite state space.\n", + "\n", + "Assume for now that $S$ has $n$ elements and let $P$ be a **Markov matrix**,\n", + "which means that $P(x,y) \\geq 0$ and $\\sum_y P(x,y) = 1$ for all $x$.\n", + "\n", + "In applications, $P(x, y)$ represents the probability of transitioning from $x$ to\n", + "$y$ in one step.\n", + "\n", + "A **Markov chain** $(X_t)_{t \\in \\ZZ_+}$ on $S$ with Markov\n", + "matrix $P$ is a sequence of random variables satisfying \n", + "\n", + "$$\n", + " \\PP\\{X_{t+1} = y \\,|\\, X_0, X_1, \\ldots, X_t \\} = P (X_t, y)\n", + "$$ (markovpropd)\n", + "\n", + "with probability one for all $y \\in S$ and any $t \\in \\ZZ_+$.\n", + "\n", + "In addition to connecting probabilities to the Markov matrix,\n", + "{eq}`markovpropd` says that the process depends on its history only through\n", + "the current state.\n", + "\n", + "We [recall that](https://python.quantecon.org/finite_markov.html), if $X_t$\n", + "has distribution $\\phi$, then $X_{t+1}$ has distribution $\\phi P$.\n", + "\n", + "Since $\\phi$ is understood as a row vector, the meaning is\n", + "\n", + "$$\n", + " (\\phi P)(y) = \\sum_x \\phi(x) P(x, y) \n", + " \\qquad (y \\in S)\n", + "$$ (update_rule)\n", + "\n", + "(jdfin)=\n", + "#### The Joint Distribution\n", + "\n", + "In general, for given Markov matrix $P$, there can be many Markov chains\n", + "$(X_t)$ that satisfy {eq}`markovpropd`.\n", + "\n", + "This is due to the more general observation that, for a given distribution\n", + "$\\phi$, we can construct many random variables having distribution $\\phi$.\n", + "\n", + "(The exercises below ask for one example.)\n", + "\n", + "Hence $P$ is, in a sense, a more primitive object than $(X_t)$.\n", + "\n", + "There is another way to see the fundamental importance of $P$, which is by\n", + "constructing the joint distribution of $(X_t)$ from $P$.\n", + "\n", + "Let $S^\\infty$ represent the space of $S$-valued sequences $(x_0, x_1, x_2, \\ldots)$.\n", + "\n", + "Fix an initial condition $\\psi \\in \\dD$ and a Markov matrix $P$ on $S$.\n", + "\n", + "The **joint distribution** of a Markov chain $(X_t)$ satisfying\n", + "{eq}`markovpropd` and $X_0 \\sim \\psi$ is the distribution $\\mathbf P_\\psi$ over\n", + "$S^\\infty$ such that\n", + "\n", + "$$\n", + " \\PP\\{ X_{t_1} = y_1, \\ldots, X_{t_k} = y_k \\}\n", + " =\n", + " \\mathbf P_\\psi\\{ (x_t) \\in S^\\infty \\,:\\, \n", + " x_{t_i} = y_i \\text{ for } i = 1, \\ldots m\\}\n", + "$$ (jointdeq)\n", + "\n", + "for any $m$ positive integers $t_i$ and $m$ elements $y_i$ of the state space $S$.\n", + "\n", + "(Joint distributions of discrete time processes are uniquely defined by their\n", + "values at finite collections of times --- see, for example, Theorem 7.2 of {cite}`walsh2012knowing`.)\n", + "\n", + "We can construct $\\mathbf P_\\psi$ by first defining $P_\\psi^n$ over \n", + "the the finite Cartiesian product $S^{n+1}$ via\n", + "\n", + "$$\n", + " \\mathbf P_\\psi^n(x_0, x_1, \\ldots, x_n)\n", + " = \\psi(x_0)\n", + " P(x_0, x_1)\n", + " \\times \\cdots \\times\n", + " P(x_{n-1}, x_n)\n", + "$$ (mathjointd)\n", + "\n", + "For any Markov chain $(X_t)$ satisfying {eq}`markovpropd` and $X_0 \\sim \\psi$,\n", + "the restriction $(X_0, \\ldots, X_n)$ has joint distribution $\\mathbf\n", + "P_\\psi^n$.\n", + "\n", + "This is a solved exercise below.\n", + "\n", + "The last step is to show that the family $(\\mathbf P_\\psi^n)$ defined at each\n", + "$n \\in \\NN$ extends uniquely to a distribution $\\mathbf P_\\psi$ over the\n", + "infinite sequences in $S^\\infty$.\n", + "\n", + "That this is true follows from a well known [theorem of Kolmogorov](https://en.wikipedia.org/wiki/Kolmogorov_extension_theorem).\n", + "\n", + "Hence $P$ defines the joint distribution $\\mathbf P_\\psi$ when paired with any initial condition $\\psi$.\n", + "\n", + "\n", + "\n", + "### Extending to Countable State Spaces\n", + "\n", + "When $S$ is infinite, the same idea carries through.\n", + "\n", + "Consistent with the finite case, a **Markov matrix** is a map\n", + "$P$ from $S \\times S$ to $\\RR_+$ satisfying\n", + "\n", + "$$\n", + " \\sum_y P(x, y) = 1 \n", + " \\text{ for all } x \\in S\n", + "$$\n", + "\n", + "The definition of a Markov chain $(X_t)_{t \\in \\ZZ_+}$ on $S$ with Markov matrix $P$ is exactly as in {eq}`markovpropd`.\n", + "\n", + "Given Markov matrix $P$ and $\\phi \\in \\dD$, we define $\\phi P$ by\n", + "{eq}`update_rule`.\n", + " \n", + "Then, as before, $\\phi P$ can be understood as the distribution of \n", + "$X_{t+1}$ when $X_t$ has distribution $\\phi$.\n", + "\n", + "The function $\\phi P$ is in $\\dD$, since, by {eq}`update_rule`, it is\n", + "nonnegative and\n", + "\n", + "$$\n", + " \\sum_y (\\phi P)(y) \n", + " = \\sum_y \\sum_x P(x, y) \\phi(x)\n", + " = \\sum_x \\sum_y P(x, y) \\phi(x)\n", + " = \\sum_x \\phi(x)\n", + " = 1\n", + "$$ \n", + "\n", + "(Swapping the order of infinite sums is justified here by the fact that all\n", + "elements are nonnegative --- a version of Tonelli's theorem).\n", + "\n", + "If $P$ and $Q$ are Markov matrices on $S$, then, using the definition in\n", + "{eq}`kernprod`, \n", + "\n", + "$$\n", + " (P Q)(x, y) := \\sum_z P(x, z) Q(z, y)\n", + "$$ \n", + "\n", + "It is not difficult to check that $P Q$ is again a Markov matrix on $S$.\n", + "\n", + "The elements of $P^k$, the $k$-th product of $P$ with itself, give $k$ step transition probabilities.\n", + "\n", + "For example, we have\n", + "\n", + "$$\n", + " P^k(x, y) \n", + " = (P^{k-j} P^j)(x, y) = \\sum_z P^{k-j}(x, z) P^j(z, y)\n", + "$$ (kernprodk)\n", + "\n", + "which is a version of the (discrete time) Chapman-Kolmogorov equation.\n", + "\n", + "Equation {eq}`kernprodk` can be obtained from the law of total probability: if\n", + "$(X_t)$ is a Markov chain with Markov matrix $P$ and initial condition $X_0 =\n", + "x$, then \n", + "\n", + "$$\n", + " \\PP\\{X_k = y\\}\n", + " = \\sum_z \\PP\\{X_k = y \\,|\\, X_j=z\\} \\PP\\{X_j=z\\}\n", + "$$\n", + "\n", + "\n", + "All of the {ref}`preceding discussion ` on the connection between $P$\n", + "and the joint distribution of $(X_t)$ when $S$ is finite carries over \n", + "to the current setting.\n", + "\n", + "\n", + "\n", + "\n", + "### The Continuous Time Case\n", + "\n", + "A **continuous time stochastic process** on $S$ is a collection $(X_t)$ of $S$-valued\n", + "random variables $X_t$ defined on a common probability space and indexed by $t\n", + "\\in \\RR_+$.\n", + "\n", + "Let $I$ be the Markov matrix on $S$ defined by $I(x,y) = \\mathbb 1\\{x = y\\}$.\n", + "\n", + "A **Markov semigroup** is a family $(P_t)$ of Markov matrices\n", + "on $S$ satisfying \n", + "\n", + "1. $P_0 = I$,\n", + "2. $\\lim_{t \\to 0} P_t(x, y) = I(x,y)$ for all $x,y$ in $S$, and\n", + "3. the semigroup property $P_{s + t} = P_s P_t$ for all $s, t \\geq 0$.\n", + "\n", + "The interpretation of $P_t(x, y)$ is the probability of moving from state $x$\n", + "to state $y$ in $t$ units of time.\n", + "\n", + "As such it is natural that $P_0(x,y) = 1$ if $x=y$ and zero otherwise, which\n", + "is condition 1.\n", + "\n", + "Condition 2 is continuity with respect to $t$, which might seem restrictive\n", + "but it is in fact very mild.\n", + "\n", + "For all practical applications, probabilities do not jump --- although the\n", + "chain $(X_t)$ itself can of course jump from state to state as time\n", + "goes by.[^footnote1] \n", + "\n", + "[^footnote1]: On a technical level, right continuity of paths for $(X_t)$ implies condition 2, as proved in Theorem 2.12 of {cite}`liggett2010continuous`. Right continuity of paths allows for jumps, but insists on only finitely many jumps in any bounded interval.\n", + "\n", + "\n", + "The semigroup property in condition 3 is nothing more than a continuous\n", + "time version of the Chapman-Kolmogorov equation.\n", + "\n", + "This becomes clearer if we write it more explicitly as\n", + "\n", + "$$\n", + " P_{s+t}(x, y) \n", + " = \\sum_z P_s(x, z) P_t(z, y)\n", + "$$ (chapkol_ct2)\n", + "\n", + "A stochastic process $(X_t)$ is called a (time homogeneous) **continuous time\n", + "Markov chain** on $S$ with Markov semigroup $(P_t)$ if\n", + "\n", + "$$\n", + " \\PP\\{X_{s + t} = y \\,|\\, \\fF_s \\}\n", + " = P_t (X_s, y)\n", + "$$ (markovprop)\n", + "\n", + "with probability one for all $y \\in S$ and $s, t \\geq 0$.\n", + "\n", + "Here $\\fF_s$ is the history $(X_r)_{r \\leq s}$ of the process up until\n", + "time $s$.\n", + "\n", + "If you are an economist you might call $\\fF_s$ the \"information set\" at time\n", + "$s$.\n", + "\n", + "If you are familiar with measure theory, you can understand $\\fF_s$ as\n", + "the $\\sigma$-algebra generated by $(X_r)_{r \\leq s}$.\n", + "\n", + "Analogous to the discrete time case, the joint\n", + "distribution of $(X_t)$ is determined by its Markov semigroup plus an\n", + "initial condition.\n", + "\n", + "This distribution is defined over the set of all right continuous functions\n", + "$\\RR_+ \\ni t \\mapsto x_t \\in S$, which we call $rcS$.\n", + "\n", + "Next one builds finite dimensional distributions over $rcS$ using\n", + "expressions similar to {eq}`mathjointd`.\n", + "\n", + "Finally, the Kolmogorov extension theorem is applied, similar to the discrete\n", + "time case.\n", + "\n", + "Corollary 6.4 of {cite}`le2016brownian` provides full details.\n", + "\n", + "\n", + "### The Canonical Chain\n", + "\n", + "Given a Markov semigroup $(P_t)$ on $S$, does there always exist a continuous\n", + "time Markov chain $(X_t)$ such that {eq}`markovprop` holds?\n", + "\n", + "The answer is affirmative.\n", + "\n", + "To illustrate, pick any Markov semigroup $(P_t)$ on $S$ and fix initial\n", + "condition $\\psi$.\n", + "\n", + "Next, create the corresponding joint distribution $\\mathbf P_\\psi$ over\n", + "$rcS$, as described above.\n", + "\n", + "Now, for each $t \\geq 0$, let $\\pi_t$ be the point evaluation functional on\n", + "$rcS$, that maps right continuous function $(x_\\tau)$ into its time $t$ value\n", + "$x_t$.\n", + "\n", + "Finally, let $X_t$ be an $S$-valued function on $rcS$ defined at $(x_\\tau) \\in rcS$ by $\\pi_t ( (x_\\tau))$.\n", + "\n", + "In other words, after $\\mathbf P_\\psi$ picks out some time path $(x_\\tau) \\in\n", + "rcS$, the Markov chain $(X_t)$ simply reports this time path.\n", + "\n", + "Hence $(X_t)$ automatically has the correct distribution.\n", + "\n", + "The chain $(X_t)$ constructed in this way is called the **canonical chain**\n", + "for the semigroup $(P_t)$ and initial condition $\\psi$.\n", + "\n", + "\n", + "### Simulation and Probabilistic Constructions\n", + "\n", + "While we have answered the existence question in the affirmative, \n", + "the canonical construction is quite abstract.\n", + "\n", + "Moreover, there is little information about how we might simulate such a chain.\n", + "\n", + "Fortunately, it turns out that there are more concrete ways to build\n", + "continuous time Markov chains from the objects that describe their\n", + "distributions.\n", + "\n", + "We will learn about these in a {doc}`later lecture `.\n", + "\n", + "\n", + "\n", + "## Implications of the Markov Property\n", + "\n", + "The Markov property carries some strong implications that are not immediately\n", + "obvious.\n", + "\n", + "Let's take some time to explore them.\n", + "\n", + "### Example: Failure of the Markov Property\n", + "\n", + "Let's look at how the Markov property can fail, via an intuitive rather than\n", + "formal discussion.\n", + "\n", + "Let $(X_t)$ be a continuous time stochastic process with state space $S = \\{0, 1\\}$.\n", + "\n", + "The process starts at $0$ and updates at follows:\n", + "\n", + "1. Draw $W$ independently from a fixed Pareto distribution.\n", + "1. Hold $(X_t)$ in its current state for $W$ units of time and then switch\n", + " to the other state.\n", + "1. Go to step 1.\n", + "\n", + "What is the probability that $X_{s+h} = i$ given both the history $(X_r)_{r \\leq s}$ and current information $X_s = i$?\n", + "\n", + "If $h$ is small, then this is close to the\n", + "probability that there are zero switches over the time interval $(s, s+h]$.\n", + "\n", + "To calculate this probability, it would be helpful to know how long the\n", + "state has been at current state $i$.\n", + "\n", + "This is because the Pareto distribution {ref}`is not memoryless `.\n", + "\n", + "(With a Pareto distribution, if we know that $X_t$ has been at $i$ for a long\n", + "time, then a switch in the near future becomes more likely.)\n", + "\n", + "As a result, the history prior to $X_s$ is useful for predicting $X_{s+h}$,\n", + "even when we know $X_s$.\n", + "\n", + "Thus, the Markov property fails.\n", + "\n", + "\n", + "\n", + "### Restrictions Imposed by the Markov Property\n", + "\n", + "From the discussion above, we see that, for continuous time Markov chains,\n", + "the length of time between jumps must be memoryless.\n", + "\n", + "Recall that, by {proof:ref}`exp_unique`, the only memoryless\n", + "distribution supported on $\\RR_+$ is the exponential distribution.\n", + "\n", + "Hence, a continuous time Markov chain waits at states for an\n", + "exponential amount of time and then jumps.\n", + "\n", + "The way that the new state is chosen must also satisfy the Markov property,\n", + "which adds another restriction. \n", + "\n", + "In summary, we already understand the following about continuous time Markov chains:\n", + "\n", + "1. Holding times are independent exponential draws.\n", + "1. New states are chosen in a ``Markovian'' way, independent of the past given the current state.\n", + "\n", + "We just need to clarify the details in these steps to have a complete description.\n", + "\n", + "\n", + "\n", + "\n", + "## Examples of Markov Processes\n", + "\n", + "Let's look at some examples of processes that possess the Markov property.\n", + "\n", + "### Example: Poisson Processes\n", + "\n", + "The Poisson process discussed in our {doc}`previous lecture ` is a\n", + "Markov process on state space $\\ZZ_+$.\n", + "\n", + "To obtain the Markov semigroup, we observe that, for $k \\geq j$,\n", + "\n", + "$$\n", + " \\PP\\{N_{s + t} = k \\,|\\, N_s = j\\}\n", + " = \\PP\\{N_{s + t} - N_s = k - j \\,|\\, N_s = j\\}\n", + " = \\PP\\{N_{s + t} - N_s = k - j\\}\n", + "$$\n", + "\n", + "where the last step is due to independence of increments.\n", + "\n", + "From stationarity of increments we have\n", + "\n", + "$$\n", + " \\PP\\{N_{s + t} - N_s = k - j\\}\n", + " = \\PP\\{N_t = k - j\\}\n", + " = e^{-\\lambda t} \\frac{ (\\lambda t)^{k-j} }{(k-j)!}\n", + "$$\n", + "\n", + "In summary, the Markov semigroup is\n", + "\n", + "$$\n", + " P_t(j, k) \n", + " = e^{-\\lambda t} \\frac{ (\\lambda t)^{k-j} }{(k-j)!} \n", + "$$ (poissemi)\n", + "\n", + "whenever $j \\leq k$ and $P_t(j, k) = 0$ otherwise.\n", + "\n", + "This chain of equalities was obtained with $N_s = j$ for arbitrary $j$, so we\n", + "can replace $j$ with $N_s$ in {eq}`poissemi` to verify the Markov property {eq}`markovprop` for the Poisson process.\n", + "\n", + "Under {eq}`poissemi`, each $P_t$ is a Markov matrix and $(P_t)$ is a\n", + "Markov semigroup.\n", + "\n", + "The proof of the semigroup property is a solved exercise below.[^footnote2]\n", + "\n", + "[^footnote2]: In the definition of $P_t$ in {eq}`poissemi`, we use the convention that $0^0 = 1$, which leads to $P_0 = I$ and $\\lim_{t \\to 0} P_t(j, k) = I(j,k)$ for all $j,k$. These facts, along with the semigroup property, imply that $(P_t)$ is a valid Markov semigroup.\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "(inventory_dynam)=\n", + "## A Model of Inventory Dynamics\n", + "\n", + "\n", + "Let $X_t$ be the inventory of a firm at time $t$, taking values in the\n", + "integers $0, 1, \\ldots, b$.\n", + "\n", + "If $X_t > 0$, then a customer arrives after $W$\n", + "units of time, where $W \\sim E(\\lambda)$ for some fixed $\\lambda > 0$.\n", + "\n", + "Upon arrival, each customer purchases $\\min\\{U, X_t\\}$ units, where $U$ is an\n", + "IID draw from the geometric distribution started at 1 rather than 0:\n", + "\n", + "$$\n", + " \\PP\\{U = k\\} = (1-\\alpha)^{k-1} \\alpha\n", + " \\qquad (k = 1, 2, \\ldots, \\; \\alpha \\in (0, 1))\n", + "$$\n", + "\n", + "If $X_t = 0$, then no customers arrive and the firm places an order for $b$ units.\n", + "\n", + "The order arrives after a delay of $D$ units of time, where $D \\sim E(\\lambda)$.\n", + "\n", + "(We use the same $\\lambda$ here just for convenience, to simplify the exposition.)\n", + "\n", + "### Representation\n", + "\n", + "The inventory process jumps to a new value either when a new customer arrives\n", + "or when new stock arrives.\n", + "\n", + "Between these arrival times it is constant.\n", + "\n", + "Hence, to track $X_t$, it is enough to track the jump times and the new values\n", + "taken at the jumps.\n", + "\n", + "In what follows, we denote the jump times by $\\{J_k\\}$ and the values at jumps\n", + "by $\\{Y_k\\}$.\n", + "\n", + "Then we construct the state process via\n", + "\n", + "$$\n", + " X_t = \\sum_{k \\geq 0} Y_k \\mathbb 1\\{J_k \\leq t < J_{k+1}\\}\n", + " \\qquad (t \\geq 0)\n", + "$$ (xfromy)\n", + "\n", + "\n", + "\n", + "### Simulation\n", + "\n", + "Let's simulate this process, starting at $X_0 = 0$.\n", + "\n", + "As above,\n", + "\n", + "* $J_k$ is the time of the $k$-th jump (up or down) in inventory.\n", + "* $Y_k$ is the size of the inventory after the $k$-th jump.\n", + "* $(X_t)$ is defined from these objects via {eq}`xfromy`.\n", + "\n", + "Here's a function that generates and returns one path $t \\mapsto X_t$.\n", + "\n", + "(We are not aiming for computational efficiency at this stage.)" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "def sim_path(T=10, seed=123, λ=0.5, α=0.7, b=10):\n", + " \"\"\"\n", + " Generate a path for inventory starting at b, up to time T.\n", + "\n", + " Return the path as a function X(t) constructed from (J_k) and (Y_k).\n", + " \"\"\"\n", + "\n", + " J, Y = 0, b\n", + " J_vals, Y_vals = [J], [Y]\n", + " np.random.seed(seed)\n", + "\n", + " while True:\n", + " W = np.random.exponential(scale=1/λ) # W ~ E(λ)\n", + " J += W\n", + " J_vals.append(J)\n", + " if J >= T:\n", + " break\n", + " # Update Y\n", + " if Y == 0:\n", + " Y = b\n", + " else:\n", + " U = np.random.geometric(α)\n", + " Y = Y - min(Y, U)\n", + " Y_vals.append(Y)\n", + " \n", + " Y_vals = np.array(Y_vals)\n", + " J_vals = np.array(J_vals)\n", + "\n", + " def X(t):\n", + " if t == 0.0:\n", + " return Y_vals[0]\n", + " else:\n", + " k = np.searchsorted(J_vals, t)\n", + " return Y_vals[k-1]\n", + "\n", + " return X" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's plot the process $(X_t)$." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAUDklEQVR4nO3df7BcZX3H8fe3EAzUpAJJ25CQJmasAzIl0jtFwTqOPzpKNbGOrYVCk6JNOxSLtrTFwRZr25naSkfrWPQWBX9Qi/FHCSgUxp9jJ9IGGhESFKVKL0SIUbxUGwH99o89sZfL3eTsvXvOuXuf92tmZ/eec/acb549+8nZZ8+eJzITSVI5fqzrAiRJ7TL4JakwBr8kFcbgl6TCGPySVJjDuy6gjmXLluWaNWu6LkOSRsott9zyzcxcPn36SAT/mjVr2LFjR9dlSNJIiYivzzTdrh5JKozBL0mFMfglqTAGvyQVxuCXpMI0FvwR8e6IeCAibp8y7ZiIuCki7qruj25q+5KkmTV5xH8l8MJp0y4CPpGZTwE+Uf0tSWpRY+fxZ+ZnI2LNtMkbgedUj98DfBr4k6Zq+PNr72DXfZNNrX4gG9ev5KxTV3ddhqQB/dPN93DNzntnnDeq7+u2+/h/KjP3AFT3P9lvwYjYEhE7ImLH3r17WyuwCbv2TPbdcSTNb9fsvJddex5/ADnK7+t5+8vdzBwHxgHGxsZmNVrMJS952lBrmq1XvHN71yVImoMTVyzl6t955mOmjfL7uu0j/vsjYgVAdf9Ay9uXpOK1HfzbgE3V403ANS1vX5KK1+TpnB8AtgNPjYiJiHgl8NfACyLiLuAF1d+SpBY1eVbPmX1mPa+pbUqSDs1f7kpSYQx+SSqMwS9JhTH4JakwBr8kFcbgl6TCGPySVBiDX5IKY/BLUmEMfkkqjMEvSYUx+CWpMAa/JBVm3o7AtdDs2jN5yBF7RnX8TqlUdd7Xg2ojBwz+Fmxcv/KQyxwY09Pgl0ZDnff1oNrKAYO/BWeduvqQL+Qoj98plajO+3pQbeWAffySVBiDX5IKY/BLUmEMfkkqjMEvSYUx+CWpMAa/JBXG4Jekwhj8klQYg1+SCmPwS1JhDH5JKozBL0mFMfglqTAGvyQVppPgj4jXRsQdEXF7RHwgIhZ3UYcklaj14I+IlcDvA2OZeRJwGPDrbdchSaXqagSuw4EjI+IR4Cjgvo7qmFeaGL9zrhwHWGrX9Bw48bilXPKSpw11G60Hf2beGxFvBu4B/he4MTNvnL5cRGwBtgCsXr3wg6eJ8TvnynGApXa1lQOtB39EHA1sBNYCDwJbI+LszHz/1OUycxwYBxgbG8u262xbE+N3ztV8+/QhLXRt5UAXX+4+H/ivzNybmY8AHwFO66AOSSpSF8F/D/CMiDgqIgJ4HrC7gzokqUitB39m3gx8CLgV+GJVw3jbdUhSqTo5qyczLwEu6WLbklQ6f7krSYUx+CWpMAa/JBXG4Jekwhj8klQYg1+SCmPwS1JhDH5JKozBL0mFMfglqTAGvyQVxuCXpMIY/JJUmK7G3NWImI/jAKsZjq9cDoNffc3HcYDVDMdXLovBr77m4zjAaoaf6spiH78kFcbgl6TCGPySVBiDX5IKY/BLUmEMfkkqjMEvSYUx+CWpMAa/JBXG4Jekwhj8klQYg1+SCmPwS1JhDH5JKkwnwR8RT4qID0XEnRGxOyKe2UUdklSirq7H/1bghsx8eUQcARzVUR2SVJxawR8Rx2Tmt4axwYhYCjwb2AyQmQ8DDw9j3ZJmb5jDbDqM4/xWt6vn5ojYGhFnRETMcZtPBvYCV0TEf0bE5RHx49MXiogtEbEjInbs3bt3jpuUdDAb16/kxBVLh7KuXXsmuWbnvUNZl5pRt6vnZ4HnA+cCb4uIq4ErM/PLs9zmKcCrM/PmiHgrcBHwp1MXysxxYBxgbGwsZ7EdSTUNc5hNh3Gc/2od8WfPTZl5JvAqYBPw7xHxmVl8MTsBTGTmzdXfH6L3H4EkqQV1+/iPBc4GzgHuB14NbAPWA1uBtXU3mJnfiIj/joinZuaXgOcBuwYtXJI0O3W7erYD7wNempkTU6bviIh3zGK7rwauqs7ouRv4rVmsQ5I0C4cM/og4DLguM/9ipvmZ+aZBN5qZO4GxQZ8nSZq7Q/bxZ+YPgJNbqEWS1IK6XT07I2Ibvf787x6YmJkfaaQqSVJj6gb/McA+4LlTpiVg8EvSiKkV/Jnpl6+StEDUOo8/IlZFxEcj4oGIuD8iPhwRq5ouTpI0fHUv2XAFvfP2jwNWAtdW0yRJI6Zu8C/PzCsy89HqdiWwvMG6JEkNqRv834yIsyPisOp2Nr0veyVJI6Zu8J8L/BrwDWAP8HL8ta0kjaS6p3Men5kbpk6IiNOBe4ZfkiSpSXWP+N9Wc5okaZ476BF/dcnl04DlEfEHU2YtBQ5rsjBJUjMO1dVzBPDEarklU6ZP0uvnlySNmIMGf2Z+BvhMRFyZmV9vqSZJI27Q8Xsdo7dddb/cfUJEjANrpj4nM5/b9xmSirRx/cqBlt+1ZxLA4G9R3eDfCrwDuBz4QXPlSBp1g47f6xi97asb/I9m5mWNViJJakXd0zmvjYjzImJFRBxz4NZoZZKkRtQ94t9U3f/RlGkJPHm45UiSmlb3evxrmy5EktSOutfjPyoiXl+d2UNEPCUiXtxsaZKkJgxyPf6H6f2KF2AC+MtGKpIkNapu8K/LzL8BHgHIzP8ForGqJEmNqRv8D0fEkfS+0CUi1gHfb6wqSVJj6p7V8wbgBuD4iLgKOB3Y3FBNkqQG1T2r58aIuAV4Br0ungsy85uNViZJakSt4I+IbcAHgG2Z+d1mS5IkNaluH/+lwC8CuyJia0S8PCIWN1iXJKkhdbt6Dlye+TDgucBvA++mNyCLJGmE1P1yl+qsnpcArwBOAd7TVFGSpObU7eO/GjiV3pk9bwc+nZk/bLIwSVIz6h7xXwGclZlDuxZ/1W20A7g3M738gyS1pG4f/w0RcVpErOGxI3C9dw7bvgDYjd8TSFKr6nb1vA9YB+zk/0fgSmBWwR8Rq4BfBv4K+IPZrEOSNDt1u3rGgBMzM4e03bcAfwws6bdARGwBtgCsXu1YnJI0LHXP478d+OlhbLC6nPMDmXnLwZbLzPHMHMvMseXLlw9j05Ik6h/xL6P3461/Z8rF2TJzwyy2eTqwISLOABYDSyPi/Zl59izWJUka0CAXaRuKzHwd8DqAiHgOcKGhL0ntGeSXu5KkBeCgwR8Rn8vMZ0XEQ1TX4j8wC8jMnNOpmJn5aeDTc1mHJGkwBw3+zHxWdd/37BtJ0mipe1aPJGmBMPglqTAGvyQVxuCXpMIY/JJUGINfkgpj8EtSYQx+SSqMwS9JhTH4JakwBr8kFcbgl6TC1L0evyQ1ZteeSV7xzu1zXs/G9Ss561SHaj0Ug19SpzauXzmU9ezaMwlg8Ndg8Evq1Fmnrh5KWA/jE0Mp7OOXpMIY/JJUGINfkgpj8EtSYQx+SSqMwS9JhTH4JakwBr8kFcbgl6TCGPySVBiDX5IKY/BLUmEMfkkqjMEvSYVpPfgj4viI+FRE7I6IOyLigrZrkKSSdXE9/keBP8zMWyNiCXBLRNyUmbs6qEWSitN68GfmHmBP9fihiNgNrAQMfklzMtchHEsZurHTEbgiYg3wdODmGeZtAbYArF698F8ISXMz1yEcSxq6sbPgj4gnAh8GXpOZk9PnZ+Y4MA4wNjaWLZcnacTMdQjHkoZu7OSsnohYRC/0r8rMj3RRgySVqouzegJ4F7A7M/+u7e1LUum6OOI/HTgHeG5E7KxuZ3RQhyQVqYuzej4HRNvblST1+MtdSSqMwS9JhTH4JakwBr8kFcbgl6TCGPySVBiDX5IKY/BLUmEMfkkqjMEvSYUx+CWpMAa/JBXG4Jekwhj8klQYg1+SCmPwS1JhDH5JKkzrI3ANyyOPPMLExAT79+/vupSBLV68mFWrVrFo0aKuS5FUoJEN/omJCZYsWcKaNWvojd8+GjKTffv2MTExwdq1a7suR1KBRrarZ//+/Rx77LEjFfoAEcGxxx47kp9UJC0MIxv8wMiF/gGjWrekhWGkg1+SNDiDX5IKY/BLUmEM/jm67LLLOO+883709+tf/3rOOeecDiuSpIMz+Odo06ZNXHvttTz44INcd911fOxjH2N8fLzrsiSpr5E9j3++OOqoozjzzDO5+OKLuf7667nppps48sgjuy5LkvpaEMH/59fewa77Joe6zhOPW8olL3larWXPPfdcTjjhBK655hrWrVv3o+nf/va3Ofroo4dalyTNlV09Q/DGN76R5cuX8+ijjz5m+mtf+9qOKpKk/hbEEX/dI/MmXHrppezfv58PfvCDXHLJJbzsZS8D4IYbbuDOO+/kzW9+MxdeeGFn9UnSdAsi+LvyyU9+kiuuuILt27ezZMkSJicn2blzJ+vXr2fZsmWcffbZnH/++V2XKUmP0UlXT0S8MCK+FBFfiYiLuqhhru655x5e9apXsXXrVpYsWQLABRdcwFve8hYAbrvtNk4++eQuS5SkGbV+xB8RhwFvB14ATAD/ERHbMnNX27XMxerVq7n77rsfM23z5s1s3rwZgGXLlnH55ZezbNkyTjjhhA4qlKSZddHV8wvAVzLzboCI+GdgIzBSwX8oGzZsYMOGDV2XIWkAu/ZM8op3bn/ctBNXLO2oomZ0Efwrgf+e8vcEcOr0hSJiC7AFekfXktSkjetXzjj9xBVL+84bVV0E/0zXJM7HTcgcB8YBxsbGHjdfkobprFNXc9apZRxkdvHl7gRw/JS/VwH3dVCHJBWpi+D/D+ApEbE2Io4Afh3YNpsVZY7mB4FRrVvSwtB68Gfmo8D5wL8Cu4EPZuYdg65n8eLF7Nu3b+RC9MCYu4sXL+66FEmF6uQHXJn5ceDjc1nHqlWrmJiYYO/evUOqqj2LFy9m1apVXZchqVAj+8vdRYsWsXbt2q7LkKSR40XaJKkwBr8kFcbgl6TCxCicFRMRe4Gvz/Lpy4BvDrGcYbGuwVjXYKxrMPO1LphbbT+TmcunTxyJ4J+LiNiRmWNd1zGddQ3GugZjXYOZr3VBM7XZ1SNJhTH4JakwJQT/eNcF9GFdg7GuwVjXYOZrXdBAbQu+j1+S9FglHPFLkqYw+CWpMAsm+A81gHv0/H01/7aIOKWFmo6PiE9FxO6IuCMiLphhmedExHciYmd1+7Om66q2+7WI+GK1zR0zzO+ivZ46pR12RsRkRLxm2jKttFdEvDsiHoiI26dMOyYiboqIu6r7o/s896D7YgN1/W1E3Fm9Th+NiCf1ee5BX/MG6npDRNw75bU6o89z226vq6fU9LWI2NnnuU2214zZ0No+lpkjfwMOA74KPBk4AvgCcOK0Zc4Arqc3AtgzgJtbqGsFcEr1eAnw5Rnqeg5wXQdt9jVg2UHmt95eM7ym36D3A5TW2wt4NnAKcPuUaX8DXFQ9vgh402z2xQbq+iXg8Orxm2aqq85r3kBdbwAurPE6t9pe0+ZfCvxZB+01Yza0tY8tlCP+Hw3gnpkPAwcGcJ9qI/De7Pk88KSIWNFkUZm5JzNvrR4/RG/8gVEZvLP19prmecBXM3O2v9iek8z8LPCtaZM3Au+pHr8HeOkMT62zLw61rsy8MXvjXAB8nt6odq3q0151tN5eB0REAL8GfGBY26vrINnQyj62UIJ/pgHcpwdsnWUaExFrgKcDN88w+5kR8YWIuD4intZSSQncGBG3RG9g++k6bS96I7P1e0N20V4AP5WZe6D3xgV+coZlum63c+l9UpvJoV7zJpxfdUG9u0+3RZft9YvA/Zl5V5/5rbTXtGxoZR9bKMFfZwD3WoO8NyEingh8GHhNZk5Om30rve6Mk4G3Af/SRk3A6Zl5CvAi4Pci4tnT5nfZXkcAG4CtM8zuqr3q6rLdLgYeBa7qs8ihXvNhuwxYB6wH9tDrVpmus/YCzuTgR/uNt9chsqHv02aYNlCbLZTgrzOAeyeDvEfEInov7FWZ+ZHp8zNzMjP/p3r8cWBRRCxruq7MvK+6fwD4KL2Pj1N10l6VFwG3Zub902d01V6V+w90d1X3D8ywTFf72SbgxcBvZNURPF2N13yoMvP+zPxBZv4Q+Mc+2+uqvQ4HXgZc3W+ZpturTza0so8tlOCvM4D7NuA3q7NVngF858BHqqZUfYjvAnZn5t/1Weanq+WIiF+g95rsa7iuH4+IJQce0/ty8PZpi7XeXlP0PRLror2m2AZsqh5vAq6ZYZk6++JQRcQLgT8BNmTm9/osU+c1H3ZdU78T+pU+22u9vSrPB+7MzImZZjbdXgfJhnb2sSa+se7iRu8slC/T+7b74mra7wK/Wz0O4O3V/C8CYy3U9Cx6H8FuA3ZWtzOm1XU+cAe9b+Y/D5zWQl1Prrb3hWrb86K9qu0eRS/If2LKtNbbi95/PHuAR+gdYb0SOBb4BHBXdX9MtexxwMcPti82XNdX6PX5HtjH3jG9rn6vecN1va/ad26jF0wr5kN7VdOvPLBPTVm2zfbqlw2t7GNeskGSCrNQunokSTUZ/JJUGINfkgpj8EtSYQx+SSqMwS9NExFPiojzqsfHRcSHuq5JGiZP55Smqa6dcl1mntRxKVIjDu+6AGke+mtgXXWd9ruAEzLzpIjYTO9qiYcBJ9G79swRwDnA94EzMvNbEbGO3o/flgPfA347M+9s/58hzcyuHunxLqJ3Sej1wB9Nm3cScBa967b8FfC9zHw6sB34zWqZceDVmfnzwIXAP7RStVSTR/zSYD6VveunPxQR3wGuraZ/Efi56mqLpwFbq0sKATyh/TKl/gx+aTDfn/L4h1P+/iG999OPAQ9WnxakecmuHunxHqI3HN7AsndN9f+KiF+FH41dfPIwi5PmyuCXpsnMfcC/VQN0/+0sVvEbwCsj4sCVHYc2lKA0DJ7OKUmF8Yhfkgpj8EtSYQx+SSqMwS9JhTH4JakwBr8kFcbgl6TC/B/ivnpBTVUjrwAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/markov_prop_5_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "T = 20\n", + "X = sim_path(T=T)\n", + "\n", + "grid = np.linspace(0, T, 100)\n", + "\n", + "fig, ax = plt.subplots()\n", + "ax.step(grid, [X(t) for t in grid], label=\"$X_t$\")\n", + "\n", + "ax.set(xlabel=\"time\", ylabel=\"inventory\")\n", + "\n", + "ax.legend()\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As expected, inventory falls and then jumps back up to $b$.\n", + "\n", + "\n", + "\n", + "### The Embedded Jump Chain\n", + "\n", + "In models such as the one described above, the embedded discrete time \n", + "process $(Y_k)$ is called the \"embedded jump chain\".\n", + "\n", + "It is easy to see that $(Y_k)$ is discrete time finite state Markov chain.\n", + "\n", + "Its Markov matrix $K$ is\n", + "given by $K(x, y) = \\mathbb 1\\{y=b\\}$ when $x=0$ and, when $0 < x \\leq b$, \n", + "\n", + "$$\n", + " K(x, y)\n", + " =\n", + " \\begin{cases}\n", + " \\mathbb 0 & \\text{ if } y \\geq x\n", + " \\\\\n", + " \\PP\\{x - U = y\\} = (1-\\alpha)^{x-y-1} \\alpha \n", + " & \\text{ if } 0 < y < x\n", + " \\\\\n", + " \\PP\\{U \\geq x\\} = (1-\\alpha)^{x-1}\n", + " & \\text{ if } y = 0\n", + " \\end{cases}\n", + "$$ (ijumpkern)\n", + "\n", + "\n", + "\n", + "\n", + "### Markov Property\n", + "\n", + "The inventory model just described has the Markov property precisely because\n", + "\n", + "1. the jump chain $(Y_k)$ is Markov in discrete time and\n", + "1. the holding times are independent exponential draws.\n", + "\n", + "Rather than providing more details on these points here, let us first describe\n", + "a more general setting where the arguments will be clearer and more useful.\n", + "\n", + "\n", + "\n", + "## Jump Processes with Constant Rates\n", + "\n", + "The examples we have focused on so far are special cases of Markov processes\n", + "with constant jump intensities.\n", + "\n", + "This processes turn out to be very representative (although the constant jump intensity will later be relaxed).\n", + "\n", + "Let's now summarize the model and its properties.\n", + "\n", + "\n", + "### Construction\n", + "\n", + "The data for a Markov process on $S$ with constant jump rates are \n", + "\n", + "* a parameter $\\lambda > 0$ called the **jump rate**, which governs the jump\n", + " intensities and\n", + "* a Markov matrix $K$ on $S$, called the **jump matrix**.\n", + "\n", + "To run the process we also need an initial condition $\\psi \\in \\dD$.\n", + "\n", + "The process $(X_t)$ is constructed by holding at each state for an\n", + "exponential amount of time, with rate $\\lambda$, and then updating to a\n", + "new state via $K$.\n", + "\n", + "In more detail, the construction is\n", + "\n", + "```{proof:algorithm} Constant Rate Jump Chain\n", + "\n", + "**Inputs** $\\psi \\in \\dD$, positive constant $\\lambda$, Markov matrix $K$\n", + "\n", + "**Outputs** Markov chain $(X_t)$\n", + "\n", + "1. draw $Y_0$ from $\\psi$ \n", + "1. set $k = 1$ and $J_0 = 0$\n", + "1. draw $W_k$ from Exp$(\\lambda)$ and set $J_k = J_{k-1} + W_k$\n", + "1. set $X_t = Y_{k-1}$ for all $t$ such that $J_{k-1} \\leq t < J_k$.\n", + "1. draw $Y_k$ from $K(Y_{k-1}, \\cdot)$ \n", + "1. set $k = k+1$ and go to step 3.\n", + "\n", + "```\n", + "\n", + "An alternative, more parsimonious way to express the same process is to take \n", + "\n", + "* $(N_t)$ to be a Poisson process with rate $\\lambda$ and\n", + "* $(Y_k)$ to be a discrete time Markov chain with Markov matrix $K$\n", + "\n", + "and then set\n", + "\n", + "$$\n", + " X_t := Y_{N_t} \\text{ for all } t \\geq 0\n", + "$$\n", + "\n", + "As before, the discrete time process $(Y_k)$ is called the **embedded jump chain**.\n", + "\n", + "(Not to be confused with $(X_t)$, which is often called a \"jump process\" or\n", + "\"jump chain\" due to the fact that it changes states with jumps.)\n", + "\n", + "The draws $(W_k)$ are called the **wait times** or **holding times**.\n", + "\n", + "\n", + "### Examples\n", + "\n", + "The Poisson process with rate $\\lambda$ is a jump process on $S = \\ZZ_+$.\n", + "\n", + "The holding times are obviously exponential with constant rate $\\lambda$.\n", + "\n", + "The jump matrix is just $K(i, j) = \\mathbb 1\\{j = i+1\\}$, so that the state\n", + "jumps up by one at every $J_k$.\n", + "\n", + "The inventory model is also a jump process with constant rate $\\lambda$, this\n", + "time on $S = \\{0, 1, \\ldots, b\\}$.\n", + "\n", + "The jump matrix was given in {eq}`ijumpkern`.\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "### Markov Property\n", + "\n", + "Let's show that the jump process $(X_t)$ constructed above satisfies the\n", + "Markov property, and obtain the Markov semigroup at the same time.\n", + "\n", + "We will use two facts:\n", + "\n", + "* the jump chain $(Y_k)$ has the Markov property in discrete\n", + " time and\n", + "* the Poisson process has stationary independent increments.\n", + "\n", + "From these facts it is intuitive that the distribution of $X_{t+s}$ given\n", + "the whole history $\\fF_s = \\{ (N_r)_{r \\leq s}, (Y_k)_{k \\leq N_s} \\}$\n", + "depends only on $X_s$.\n", + "\n", + "Indeed, if we know $X_s$, then we can simply\n", + "\n", + "* {ref}`restart ` the Poisson process from $N_s$ and then \n", + "* starting from $X_s = Y_{N_s}$, update the embedded jump chain $(Y_k)$ using $K$ each time a new jump occurs. \n", + "\n", + "Let's write this more mathematically.\n", + "\n", + "Fixing $y \\in S$ and $s, t \\geq 0$, we have\n", + "\n", + "\n", + "$$\n", + " \\PP\\{X_{s + t} = y \\,|\\, \\fF_s \\}\n", + " = \\PP\\{Y_{N_{s + t}} = y \\,|\\, \\fF_s \\}\n", + " = \\PP\\{Y_{N_s + N_{s + t} - N_s} = y \\,|\\, \\fF_s \\}\n", + "$$\n", + "\n", + "{ref}`Recalling ` that $N_{s + t} - N_s$ is Poisson distributed with rate $t \\lambda$, independent of the history $\\fF_s$, we can write the display above as \n", + "\n", + "$$\n", + " \\PP\\{X_{s + t} = y \\,|\\, \\fF_s \\}\n", + " =\n", + " \\sum_{k \\geq 0}\n", + " \\PP\\{Y_{N_s + k} = y \\,|\\, \\fF_s \\}\n", + " \\frac{(t \\lambda )^k}{k!} e^{-t \\lambda}\n", + "$$\n", + "\n", + "Because the embedded jump chain is Markov with Markov matrix $K$, we can simplify further to\n", + "\n", + "$$\n", + " \\PP\\{X_{s + t} = y \\,|\\, \\fF_s \\}\n", + " = \\sum_{k \\geq 0}\n", + " K^k(Y_{N_s}, y) \\frac{(t \\lambda )^k}{k!} e^{-t \\lambda}\n", + " = K^k(X_s, y) \\frac{(t \\lambda )^k}{k!} e^{-t \\lambda}\n", + "$$\n", + "\n", + "Since the expression above depends only on $X_s$,\n", + "we have proved that $(X_t)$ has the Markov property.\n", + "\n", + "\n", + "(consjumptransemi)=\n", + "### Transition Semigroup\n", + "\n", + "The Markov semigroup can be obtained from our final result, conditioning\n", + "on $X_s = x$ to get\n", + "\n", + "$$\n", + " P^t(x, y) = \\PP\\{X_{s + t} = y \\,|\\, X_s = x \\}\n", + " = e^{-t \\lambda} \\sum_{k \\geq 0}\n", + " K^k(x, y) \\frac{(t \\lambda )^k}{k!} \n", + "$$\n", + "\n", + "If $S$ is finite, we can write this in matrix form and use the definition of\n", + "the [matrix exponential](https://en.wikipedia.org/wiki/Matrix_exponential) to\n", + "get\n", + "\n", + "$$\n", + " P^t \n", + " = e^{-t \\lambda}\n", + " \\sum_{k \\geq 0}\n", + " \\frac{(t \\lambda K)^k}{k!} \n", + " = e^{-t \\lambda} e^{t \\lambda K}\n", + " = e^{t \\lambda (K - I)}\n", + "$$\n", + "\n", + "This is a simple and elegant representation of the Markov semigroup that\n", + "makes it easy to understand and analyze distribution dynamics.\n", + "\n", + "For example, if $X_0$ has distribution $\\psi$, then $X_t$ has distribution\n", + "\n", + "$$\n", + " \\psi P_t = \\psi e^{t \\lambda (K - I)}\n", + "$$ (distflowconst)\n", + "\n", + "We just need to plug in $\\lambda$ and $K$ to obtain the entire flow $t \\mapsto \\psi P_t$.\n", + "\n", + "We will soon extend this representation to the case where $S$ is infinite.\n", + "\n", + "\n", + "(invdistflows)=\n", + "## Distribution Flows for the Inventory Model\n", + "\n", + "Let's apply these ideas to the inventory model described above.\n", + "\n", + "We fix \n", + "\n", + "* the parameters $\\alpha$, $b$ and $\\lambda$ in the inventory model and\n", + "* an initial condition $X_0 \\sim \\psi_0$, where $\\psi_0$ is an arbitrary\n", + "distribution on $S$.\n", + "\n", + "The state $S$ is set to $\\{0, \\ldots, b\\}$ and the matrix $K$ is defined by\n", + "{eq}`ijumpkern`.\n", + "\n", + "Now we run time forward.\n", + "\n", + "We are interesting in computing the flow of distributions $t \\mapsto \\psi_t$,\n", + "where $\\psi_t$ is the distribution of $X_t$.\n", + "\n", + "According to the theory developed above, we have two options:\n", + "\n", + "Option 1 is to use simulation.\n", + "\n", + "The first step is to simulate many independent observations the process $(X_t^m)_{m=1}^M$.\n", + "\n", + "(Here $m$ indicates simulation number $m$, which you might think of as the outcome\n", + "for firm $m$.)\n", + "\n", + "Next, for any given $t$, we define $\\hat \\psi_t \\in \\dD$ as the\n", + "histogram of observations at time $t$, or, equivalently the cross-sectional\n", + "distribution at $t$:\n", + "\n", + "$$\n", + " \\hat \\psi_t(x) := \\frac{1}{M} \\sum_{m=1}^M \\mathbb 1\\{X_t = x\\}\n", + " \\qquad (x \\in S)\n", + "$$\n", + "\n", + "Then $\\hat \\psi_t(x)$ will be close to $\\PP\\{X_t = x\\}$ by the law of\n", + "large numbers.\n", + "\n", + "In other words, in the limit we recover $\\psi_t$.\n", + "\n", + "Option 2 is to insert the parameters into the right hand side of {eq}`distflowconst`\n", + "and compute $\\psi_t$ as $\\psi_0 P_t$.\n", + "\n", + "The figure below is created using option 2, with $\\alpha = 0.6$, $\\lambda = 0.5$ and $b=10$.\n", + "\n", + "For the initial distribution we pick a binomial distribution.\n", + "\n", + "\n", + "Since we cannot compute the entire uncountable flow $t \\mapsto \\psi_t$, we\n", + "iterate forward 200 steps at time increments $h=0.1$.\n", + "\n", + "In the figure, hot colors indicate initial conditions and early dates (so that the\n", + "distribution \"cools\" over time)\n", + "\n", + "```{glue:figure} flow_fig\n", + ":name: \"flow_fig\"\n", + "\n", + "Probability flows for the inventory model.\n", + "```\n", + "\n", + "In the (solved) exercises you will be asked to try to reproduce this figure.\n", + "\n", + "\n", + "## Exercises\n", + "\n", + "### Exercise 1\n", + "\n", + "Consider the binary (Bernoulli) distribution where outcomes $0$ and $1$ each have\n", + "probability $0.5$.\n", + "\n", + "Construct two different random variables with this distribution.\n", + "\n", + "\n", + "\n", + "### Exercise 2\n", + "\n", + "Show by direct calculation that the Poisson matrices $(P_t)$ defined in \n", + "{eq}`poissemi` satisfy the semigroup property {eq}`chapkol_ct2`.\n", + "\n", + "Hints\n", + "\n", + "* Recall that $P_t(j, k) = 0$ whenever $j > k$.\n", + "* Consider using the [binomial formula](https://en.wikipedia.org/wiki/Binomial_theorem).\n", + "\n", + "\n", + "### Exercise 3\n", + "\n", + "Consider the distribution over $S^{n+1}$ previously shown in {eq}`mathjointd`, which is\n", + "\n", + "$$\n", + " \\mathbf P_\\psi^n(x_0, x_1, \\ldots, x_n)\n", + " = \\psi(x_0)\n", + " P(x_0, x_1)\n", + " \\times \\cdots \\times\n", + " P(x_{n-1}, x_n)\n", + "$$ \n", + "\n", + "Show that, for any Markov chain $(X_t)$ satisfying {eq}`markovpropd`\n", + "and $X_0 \\sim \\psi$, the restriction $(X_0, \\ldots, X_n)$ has joint\n", + "distribution $\\mathbf P_\\psi^n$.\n", + "\n", + "### Exercise 4\n", + "\n", + "Try to produce your own version of the figure {ref}`flow_fig`.\n", + "\n", + "The initial condition is ``ψ_0 = binom.pmf(states, n, 0.25)`` where ``n = b + 1``.\n", + "\n", + "\n", + "## Solutions\n", + "\n", + "### Solution to Exercise 1\n", + "\n", + "This is easy.\n", + "\n", + "One example is to take $U$ to be uniform on $(0, 1)$ and set $X=0$ if $U <\n", + "0.5$ and $1$ otherwise.\n", + "\n", + "Then $X$ has the desired distribution.\n", + "\n", + "Alternatively, we could take $Z$ to be standard normal and set $X=0$ if $Z <\n", + "0$ and $1$ otherwise.\n", + " \n", + "\n", + "### Solution to Exercise 2\n", + "\n", + "Fixing $s, t \\in \\RR_+$ and $j \\leq k$, we have \n", + "\n", + "$$\n", + "\\begin{aligned}\n", + " \\sum_{i \\geq 0} P_s(j, i) P_t(i, k)\n", + " & = \n", + " e^{-\\lambda (s+t)} \n", + " \\sum_{j \\leq i \\leq k}\n", + " \\frac{ (\\lambda s)^{i-j} }{(i-j)!} \n", + " \\frac{ (\\lambda t)^{k-i} }{(k-i)!} \n", + " \\\\\n", + " & = \n", + " e^{-\\lambda (s+t)} \\lambda^{k-j}\n", + " \\sum_{0 \\leq \\ell \\leq k-j}\n", + " \\frac{ s^\\ell }{\\ell!} \n", + " \\frac{ t^{k-j - \\ell} }{(k-j - \\ell)!} \n", + " \\\\\n", + " & = \n", + " e^{-\\lambda (s+t)} \\lambda^{k-j}\n", + " \\sum_{0 \\leq \\ell \\leq k-j}\n", + " \\binom{k-j}{\\ell}\n", + " \\frac{s^\\ell t^{k-j - \\ell}}{(k-j)!} \n", + "\\end{aligned}\n", + "$$\n", + "\n", + "Applying the binomial formula, we can write this as \n", + "\n", + "$$\n", + " \\sum_{i \\geq 0} P_s(j, i) P_t(i, k)\n", + " =\n", + " e^{-\\lambda (s+t)} \n", + " \\frac{(\\lambda (s + t))^{k-j}}{(k-j)!}\n", + " = P_{s+t}(j, k)\n", + "$$\n", + "\n", + "Hence {eq}`chapkol_ct2` holds, and the semigroup property is satisfied.\n", + "\n", + "\n", + "### Solution to Exercise 3 \n", + "\n", + "Let $(X_t)$ be a Markov chain satisfying {eq}`markovpropd` and $X_0 \\sim \\psi$.\n", + "\n", + "When $n=0$, we have $\\mathbf P_\\psi^n = \\mathbf P_\\psi^0 = \\psi$, and this\n", + "agrees with the distribution of the restriction $(X_0, \\ldots, X_n) = (X_0)$.\n", + "\n", + "Now suppose the same is true at arbitrary $n-1$, in the sense that\n", + "the distribution of $(X_0, \\ldots, X_{n-1})$ is equal to $\\mathbf P_\\psi^{n-1}$ as\n", + "defined above.\n", + "\n", + "Then \n", + "\n", + "$$\n", + " \\PP \\{X_0 = x_0, \\ldots, X_n = x_n\\}\n", + " = \\PP \\{X_n = x_n \\,|\\, X_0 = x_0, \\ldots, X_{n-1} = x_{n-1} \\}\n", + " \\\\\n", + " \\times \\PP \\{X_0 = x_0, \\ldots, X_{n-1} = x_{n-1}\\}\n", + "$$\n", + "\n", + "From the Markov property and the induction hypothesis, the right hand side is\n", + "\n", + "$$\n", + " P (x_{n-1}, x_n )\n", + " \\mathbf P_\\psi^n(x_0, x_1, \\ldots, x_{n-1})\n", + " =\n", + " P (x_n, x_{n+1} )\n", + " \\psi(x_0)\n", + " P(x_0, x_1)\n", + " \\times \\cdots \\times\n", + " P(x_{n-1}, x_{n-1})\n", + "$$\n", + "\n", + "The last expression equals $\\mathbf P_\\psi^n$, which concludes the proof.\n", + "\n", + "\n", + "### Solution to Exercise 4 \n", + "\n", + "Here is one approach.\n", + "\n", + "(The statements involving ``glue`` are specific to this book and can be deleted\n", + "by most readers. They store the output so it can be displayed elsewhere.)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAAFUCAYAAACp7gyoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9eXAc133v++1lVswAxMoFoLiTICmRoiTKpBL7pm49W0tu+T7p1q34+darih0+SbEr0XXi1FPixIkc2+9ZdhRH0Y3kWJRtyZIVP8eWnNhSJMuSvMgiRVHURoLY930AYnrWnl7eH4PTON3TPdOzAGgA51OFAjDT3XNmpvt8+/c7v4XTdR0MBoPBYDCKw6/2ABgMBoPBWAswwWQwGAwGwwVMMBkMBoPBcAETTAaDwWAwXMAEk8FgMBgMFzDBZDAYDAbDBWKJ51nOCYPBYDA2Gpzdg8zCZDAYDAbDBUwwGQwGg8FwARNMBoPBYDBcwASTwWAwGAwXMMFkMBgMBsMFTDAZDAaDwXABE0wGg8FgMFzABJPBYDAYDBcwwWQwGAwGwwVMMBkMBoPBcAETTAaDwWAwXMAEk8FgMBgMFzDBZDAYDAbDBUwwGQwGg8FwARNMBoPBYDBcwASTwWAwGAwXMMFkMBgMBsMFTDAZDAaDwXABE0wGg8FgMFwgrvYAGIzVQNd1aJoGRVEgCAJ4ngfPs/tHBoPhDBNMxoZC13Xouo5cLgdVVZHJZAyh5DgOgiBAFEWTiHIct8qjZjAYXoDTdb3Y80WfZDDWEpqmIZfLQdM0cBwHXdchyzJ4nge5Doig0giCYAgpz/MQBIGJKIOxvrG9wJlgMtY9xPWqqiqAvCXJcRw0TTME0wkioNbrhAgn/UOOy2Aw1jy2FzJzyTLWLbquI5VKmVyu5Qqa3T5EPBVFQVdXF1pbW1FfXw8AJpcucesyEWUw1gdMMBnrDl3XoSgKFEXB2bNnceLEiZoG9BABpH8LgmAEEmWzWdP2xBql10aZNcpgrD2YYDLWDbquQ1VVKIoCXdcNUVopYSpmjeZyOciyXCCyLMCIwVg7MMFkrHnshJJ2w5LHlgNy/GLPA3lXrdOYaXieL3DpsnQXBsMbMMFkrFmICzSXyxUIJYEE99iJTjKZxPz8PCKRCMLh8IoKk5M1SiJ3rdsya5TBWH2YYDLWHHQuJUkRcRI7Owswk8mgt7cXiUQCjY2NiMViSKVS4DgOdXV1iEajiEQiiEQiEMXil0gpC7Mcirl07axRlu7CYKwsTDAZawprLmWpNUo6xzKXy6G/vx+xWAx79uzBwYMHoSiKsb+qqkgmk5AkCVNTU+jr64OqqgiFQoaARiIRBINB01pkrQTTDmuAEYFY16qqGhZpLpdDIpFAW1sbS3dhMJYBJpiMNYFTLmUpOI6DoigYGxvD+Pg4duzYgX379oHneWiaZlrfFAQB9fX1RooIkBemdDqNRCIBSZIwMTGBTCYDURQRiUSQTqfh9/vR2Ni46i7dbDaLiYkJNDU1mQKMyHtj6S4MRnWwwgUMT0NSRC5fvoyrrroKfr/f9USv6zpee+01aJqG9vZ27NixwxR8Q6zVSoSDWHODg4MAYLhLw+GwyaXr9/vLPnalpNNp9Pb24pprrjE9Xqr4Akt3YTAKYIULGGsHOpcSAOLxuOtoV13XMTMzg97eXqiqiqNHj6KhoaFgu2qEwefzobGxEfPz86ivr0dLSws0TUMqlYIkSYjFYhgaGkIul0MgEDAENBqNIhQKragosXQXBqM2MMFkeAqnXEp6LbIY8/Pz6OnpQSgUwrFjx3D58uWSgTvVQK9h8jxvCCP9frLZLBKJBBKJBKanp5FOpyEIQkGAkTX1ZDmpRboLs0YZGw0mmAxPUCyXElhKD3EikUigu7sbAHDo0CFDtJY7KKcUHMchGAwiGAyipaXFeFxRFCPAaGJiAolEApqmIRwOm6zRcl3QtRgvS3dhMOxhgslYVdzkUgLOwpfJZNDT04NUKoX9+/ejsbHR9Lxby7RSKhVkURTR0NBgchWT2reJRAILCwsYGxtDNpuFz+cziWixnNHlECuW7sJg5GGCyVgVysmlBGBEtRLoFJG9e/eitbXVdlJebQuzHEgeaF1dHTZv3mw8Lsuy4dIdGhpCKpUCAFOqC+0GXqmx0r8JdLpLPB7H+Pg4du3aVdDZhaW7MNYiTDAZK065uZTAkvCpqoqhoSFMTExgx44d2L9/f9F9VyJPcrkF2e/3o6mpCU1NTcZjqqoaAUYzMzMYGBiALMtQFAX9/f2GNUrnjK4E1u9SlmVjnVRRFJbuwljTMMFkrBiapkGSJCPVohwLg+M4TE5OYnZ2Ftu2bcOJEydcBcmUWvusltWyYAVBQDQaRTQaNR5LJpPo7e1FNBqFJEmYnJxEJpMxtiWWaF1d3YoEGJHPhbZGaS9Cqe4utFuXWaMML8AEk7Hs0Cki58+fx8mTJ8sKZJmensbU1BSam5tx4403wufzuX7tteSSrRYShNPa2orW1lbjcZIzmkgkMDY2hmQyCV3XbQOMlmNMxZ5j6S6MtQQTTMayYc2ldFr3cmJubg49PT3Gmt7mzZvLEkvyWmvdJesWp3GQnFE6IIrkjCYSCczPz2N4eBi5XA5+v98QUFKUvlJRqqRLjJt0F/q4LN2FsZIwwWTUHDuhpCewUhOpJEno7u4Gz/M4fPgwIpEIuru7KxKm5Y6SBWqTzlEr3AqFU84oCTAia6OpVMq0rdui9OR4tRIulu7C8AJMMBk1w6noAE2x/pSktFs6ncb+/fuxadMm4zlrlKxbVqo4+nqA4zgEAgEEAgE0Nzcbj6uqarh07YrSE2s0EAiUdWNUi/GWm+4C5IOo6EhdBsMtTDAZVVOq6AANET76eVmW0d/fj/n5eezduxctLS2OQlsuG8klu1wIgmCbM0qK0tM5o6IoGgK6Gp9LqXSXS5cuoaOjw7Cs6QAjlu7CKAUTTEbFlJtLCZgtRVVVMTg4iMnJSezcuRMHDhxwnKiqsTCXM0p2o8JxHMLhMMLhMNra2ozHc7kcJElCIpHA3NwcJElCPB5HXV2dKcCo3LXoWoyXnFvEVUsEnaW7MNzCBJNREZXkUgJ54VMUBVNTUxgeHkZ7eztOnjxZUmi9bGEyQV7C5/MZOaORSARzc3PYvXs3kskkEokEYrEYBgcHoSiKUZSeWKQrUZSe9m6wdBdGuTDBZJQFyaWUJAnNzc1lTRgkQOP8+fPYvHlzWSkiXl3DZDhDu+etOaOkKD2xRqempoyi9NYAo1rmjFqXA+woN92F53n4fD4WYLQBYILJcAUd+ZpMJjEzM2PK9StFLBZDb28vZFnGkSNHCmq+lqJS4fNqLdnlwCvjIBQL+qGL0tPnkaIoRoCRXVF6Yo2WU5Te7ZiKUSzdRdM0ZDIZlu6yAWCCySiKXYqIKIqurb14PI6enh4IgoCrr74afX19FbXb4nkeuVyu7P28JGgrgZcm5ErESRRFbNq0yRQhrWka0uk0JEnC/Pw8RkZGIMsyfD6fqYJRsaL09LFKbVMOLN1lY8EEk2FLsVxKN+7RVCqF3t5eZLNZ7N+/34iwXGnXaqk1xmonq40myOVQq8+F53mjKD0N3Wc0FoshlUoZBexpa5S+Qau1YNpRbrqLLMsIBAIIhUKsu4vHYYLJMOEml7KY6MmyjL6+Ply5csU2RaRSwfRylCwTTGeWc+J3yhklAUZ0zmgwGEQ0GoUsy8hkMisSYERTLN1lZGQEzc3NpvOIeHJYuou3YILJAFBZLiWNoigYGhrC5OQkdu3ahc7OTtuLu9I1RS9HyTLsIRHUK4kgCKivr0d9fb3xmK7ryGQykCQJmqaht7cXmUwGoigW9BldiaL0NOT8JEFDZLxA/pqyLkPQUbos3WXlYYK5wSFBC4qiuM6lFATBEExN0zA6OoqRkRF0dHSUTBGp1OJbLldutdVomEu2OF6YzDmOQygUQigUwtDQEI4cOQLAXJR+ZGQEyWQSABAOh01ro8tRlJ5GVVXTNVOq+IKqqqb1UVZ8YeVggrmBqTaXcmJiAgMDA2htbcUHPvABV8E8K72GuRK1ZL2C197ncpfGq5ZiReklSUIsFsPQ0BByuZyRM0oHGNXqvWma5rpVHUt3WV2YYG5AiEV58eJFo7pOORfS/Py8EbF4/fXXIxAIuN53NdYwN0paCeANi47gdcG0w6koPR1gZFeUPhqNoq6urqIIcKuFWQ4s3WVlYYK5gbBGvs7Pz5d1ocbjcXR3d0MURYRCIRw6dKjsMXgpSpZMEtVM7F4TTC+xFgXTDjpntKWlxXic5CRbc0ZDoZDJpWstSm/FrYVZ7phZukvtYYK5ASjVbqsUqVQKPT09yOVy2LdvHxoaGvDaa69VNBavWZhM8JYPXdeXPYVjNRFF0bYoPekzShel9/l8BQFG5LOpxsIsh3LTXeLxOFpaWowygORnI8MEcx1TSiiJ9eV0EWSzWfT19SEejxspItWy0tGuTvuRwu9jY2MQBAF1dXWGVRCNRl271pjgOrNeLMxyIHmgpOk5gfQZTSQSGBoaQiqVAgDU1dUhm83iypUrq1aUnv5N0HUdPT09qK+vN1mk1nSXjWaNMsFch7jJpQTsW20BeVfT4OAgpqamsHv3bhw8eLBmFwTP81BVtaL9amFh6rqO8fFxDA4Oor29HTfeeKNhFUiShOnpafT390NVVVM5tmg0uuzRkusNrwnmat7Y+P1+oyg9QVVVpFIpXLlyBbOzs0ZR+mAwaAowWumcUWDp5pp2FbtNd1nPxReYYK4jysmlBPInuaqqhjWlaRpGRkYwOjqK7du3u+oiUi4rXeKOjpKNxWLo7u5GY2OjUfhdlmXoum5bHJy41ubn5zE8PGxES9KWKNnWC3hlHAQvCqaXxiMIgmFV7t+/H8BSziixRicnJ5HJZIyi9OTcq6urW9acUbsc2krTXc6dO4ejR4+ayh2uVZhgrgMqyaUElvIpdV3H5OQk+vv7sXnzZtcpIpVQzRpmpS5ZWZbx5ptvQhAEHD16FOFw2NV+VteatcPG5OQkEomEEUxBJjM3NU2XCy8JgtcEaiXK4lULnTNqV5RekiSMjY0hmUxC07SCPqO18oKoqupakEuluzzwwAO4//77mWAyVp9KcymBvAjNzs5idHQUmzZtwvHjx8u64CqZEFeyxF02m8XAwADm5+dx7Nixsjuk2I3B2mFjfn4eU1NTaG5uNnL3SE1T2p273BaBF2GCWTucitLbeUH8fr9JREmN2nIoRzDtoNNdEomEKTBqLcMEc41ChPL999/HgQMHyl4zWFhYwNzcHHK5nGuri4a4SCsRzEpdq24FkwT0TE5OYtu2bRAEoWqxLIYgCLbrU1aLQNf1gsLgKx3ksZIwF3FpqvmMnHJGSYCRJEmYnZ01ckbpwDZrUXoriqLUzMskSRITTMbqYI18JROx24kgmUyip6cHiqKgsbERO3bsKFssAeeAoVJUWhrPzRqmNaDn5MmTSCaTkCSp7NerdlyCIBSkHGiaZuTtzczMGMFFdN5eNBotqxCE1/GSQHnRwlyOdmNORenJuihdlD4UCpmsUZIzWq2FSZPNZhEMBmtyrNWGCeYawSlFhATulLJU6BSRffv2obm5GV1dXRV38ljpfMpSlqldQA+w/HmY5ezP87zhot26dSuA/PdKej0uLCxgdHQUsizD7/cb265WpGS1eM2i2wiC6YTdDRw59xKJBOLxOMbHx5HNZiGKotHzVpIk1NXVVTxGr3kZqoUJpscplUtJBNMJRVEwMDCAmZmZghSRUvsWwyttuhKJBC5fvuwY0OP1WrIcxyEcDiMcDpuCi2RZhiRJkCQJU1NTSKfTRncNIqR2wUVeEiivCabXxgNUv1ZYDfS519bWZjyey+UwOjqKhYUFU1F6OsCo3KL0XvvcK4UJpkdxm0vpJHqapmF4eBhjY2PYvn07Tpw4UTC5Vipe5HVXs2JPNptFb28vEokEDhw44BiBtxZrydJuNbpYBOmuIUmSkfxOgosikUjF7u7lwmsCtZEtzHLw+XwIBoMQRRHbt28HYF5OiMViGBwcRC6XM3JGnTwhXnx/1cAE02NUmktJ70+6iGzZsqVoiki1FuZKFyAAzAE9e/bswaFDh4pOyl4TkWqw665Br03FYjEsLCzg7NmzBZWLViO4iAlmaVbTwiyGdVz0cgKBLkpPe0JIzuirr76K7du3m/YpxvPPP4977rkHqqri1KlTuPfee03PP/nkk/jKV74CAIhEInj44Ydx9OhRAMDOnTsRjUaN4gnnzp2r9iOwhQmmR6gml1JVVei6jtnZWfT29rpOEanGwlzpIurETfn6668bAT1uPp+1aGGWA702FYlEMDk5iX379plaVNEVZMikR3L2llPQvCiYXhoP4E0RB+AqLqJYUXoioo8//ji6urpw7NgxHDhwAEePHsXv/d7vYffu3QWv9+lPfxovvvgiOjo6cPz4cXz0ox81NXjYtWsXXn31VTQ2NuK5557DnXfeiTNnzhjPv/zyyzUp31kMJpgeoNpcSkmSMDAwgEAggGuvvRahUMjVvoIgVFR1h7zuSrlkSUCPpmmmgB43rHfBtMMp3SCTyRjBRaQoOMnZIyJay+AirwmmF4vBrxULsxxIzuif/umf4tKlS3jggQfw5JNPoqenBxcuXLD1TJ09exZ79+41hPRjH/sYnn32WZNg3nTTTcbfJ06cwOjoaEXjqwYmmKsIEcpLly5V1JcymUxiamoKHMfhyJEjrl0fhNWwMMvZzxrQc+HChbJdi14P+lkp6AoydIAH7VKbnp42XGq0O7fSKEmvCaYXrTkvjgmoXR5mPB43XKWdnZ3o7Oy03Y7EWhA6OjpM1qOV06dP49ZbbzX+5zgOH/nIR8BxHO666y7ceeedVY/dDiaYqwBxvZI7rbm5ubIumkwmg76+PiQSCTQ3NxesLbjFq1GybgN63LDRLMxyBcouZ09RFKP8Hx0lSVuikUikpAXiNcH02niA9Wlh0sTjcdTX15fczu4acvquXn75ZZw+fRq/+tWvjMd+/etfY9u2bZiensaHP/xhdHZ24kMf+lDlA3eACeYK4pQi4vYizuVyGBgYwOzsrBHwMjY2tuKiV82+xfYjAT2kS0qpgB43rKegn1LUSrhFUbQNLiJFIEgNXdLRhV4XpT0AXhMoL1pzXhwTUDvBdFvlp6OjAyMjI8b/o6Oj2LZtW8F277zzDk6dOoXnnnvOdJNHtm1ra8Ptt9+Os2fPMsFcq7hp4FxsclFVFcPDwxgfH8dVV11lShERBKGgg7pbqrEwK93XziKzVuixS4GpFDeFC5bz+OsFQRBQX19vshboWqbW4KJIJIJMJoNsNotwOOwJ4fSiOK13C1OSJFfer+PHj6OnpwcDAwNob2/H008/jaeeesq0zfDwMO644w488cQTRncXAEYh+mg0imQyiRdeeAGf//znqx67HUwwl5Fycymtawa0kGzduhUnTpwoOIlXw61azb7W9+9UoadWeGGiXq/QwUVbtmwBYG5PNT09jcHBQfT29sLn85nWRVdDRL0aJbtcnYGqoZZrmOTcKIYoinjooYdw8803Q1VVfPKTn8Thw4fxyCOPAADuvvtufOELX0AsFsOnPvUpY59z585hamoKt99+uzHuj3/847jllluqHrvtOJflqBuccnMpRVE0naC6rmNmZgZ9fX1oamoqmiJSaQGBavetRmyB0hV61gobxcJ0Cx1cNDk5ib179yIUCpkqF5GC4HSPx2qCi9yi67rnrDmvWpi1ssYTiYTr+IrbbrsNt912m+mxu+++2/j70UcfxaOPPlqw3+7du/H2229XN1CXMMGsISSXMpfLuRJKAm0lXrlyBd3d3QiFQq5SRCotIFCLfStJSclms0in00aXlXIDery2LsZwhv6u/H4/mpubC4KLSISutQQbHVxUKwvMiy5ZL46JUIvrbD11KgGYYNYE0kCYzqUs5yIQBAGSJKG7uxu6ruPgwYOu78qqXYdcKQuTDugRRRE33nhjxb00a31HXo0Ie8nC9Mo4CKU+V6cej3ZdNcLhsMkaraRRshfFyYtjqiWSJLmKkl0rMMGsEk3TMD8/D1mW0djYWHYuZSaTwZUrVxCPx3Ho0KGy+zauRnk7sq8bwbQL6Hn99dcrEigviRPBa2PykvVdab9Ua3CRrutG5SK6UXIgEDBZosFgsOjredE74VWXbK1wm1ayVmCCWSF0LiVxK9ENhEuRy+XQ39+PWCyGuro6dHR0VNTkeDUiXQF3glnrgJ5q100ZK0utBIrjONTV1aGurs507Gw2a6yLTkxMIJPJQBRFU5oLHVzkRWvOi2Oq5Q2gJElV5VF7DSaYZWKXIuLz+VwLD50ismPHDuzfvx/9/f2rErhTjXVUzDpdroAer1lzgDfH5BWW06Kj65i2trYaj8uybNzAkuAiUjg8mUwiFAp5yqrz0lgItRwTW8PcoBTLpSRRrqX2Hxsbw9DQELZt22ZKEVktK7GaycxOqMup0FOpu45ZmGuH1XCB+v1+NDU1mbw9JLior68Ps7OzmJiYgK7rBR1dViO9w4sWZi0FM5PJIBgM1uRYXoAJZgnc5FIWEy2SItLb24vm5mZb12S1orcaIkKLV7kVekh913In0+XqPVnt/szCtMcra4YkuCgcDqOjowPRaNTo7yhJEmZmZtDf3w9VVREKhUzrooFAYFnH5kULs1Y5mCQY0ms3BNXABNOBcnIpnSzM+fl5dHd3o66uDtddd53jnZYgCMhmsxWNc7UmJOKSHRsbK7tCDxHbci8kL1qYXhJMr4yD4LXx0OecU3/HdDptCi6SZdnUJDkajZYMLqp0TF6h1iLuhZumWsEE00IluZRWC1GSJPT09IDjOBw+fNjUZskOURSNHLS1wsLCAmKxGILBYEUtt1ayl+ZGwmuTk5fGU8ri5TgO4XAY4XAYmzdvNvahO7pMTk6agovoykWVdnRZr4K5Hq9VJpiLVJNLSSzMdDqN3t5epNNp7Nu3z3XUazUu2ZWGBPQAQDQadWzXU4yV7KVZCk3TqrrLZyJeHC8JZiXfs1OT5FwuZ3R0GRoaQiqVAsdxBZWLvOZudUOtBDOVSpkim9cDTDCx1InB5/OV3UEEyF88mUwGFy5cwJ49e9Da2lrW/qspmMTaKzWRWAN6wuFwxeWoqqlDW6k4Wa0LXdcxMTGB/v5+4/Fy21cx1ha1dH/6fL6C4CI6xWxsbAzJZBK6rps6ukQikZrXS641te6FuZ7Y0IJJcikVRcGbb76JkydPliV0qqpiaGgIExMT4HkeJ06cqOiOuhaCWelkUGo90SmgR1VVT7UGKwYRWvLdLCwsoKurC9FoFDfccAOAvICSIJCJiQkkEglTJGV9fb2tiDILc+2w3EFIgiCgoaHBlEZBOrqQ4KKBgQEoioJQKIRIJAJFUZDNZuH3+z1jjdeyU8l6KloAbFDBtKaIELFwe8JqmoaxsTEMDw8bwS5nzpyp+ISvVjBJikclgummU4pdQE+1VYIqEZlKxYm8XiaTQXd3N2RZNtaWVVVFLpdzbF+VTCYRj8cxOTkJSZJMIkrWrhhrg9UIsKE7umzduhXAUnAR6Sl66dIlyLIMv99vskRDodCqiKiqqhWVHrTitrXXWmJDCaabvpSl9p+enkZfXx9aWloKgl0qvYOthWDaiZ7bfa1Wm5sKPdVcyKuxhkly8Pbt24eWlhZX47eLpKTTEaampox1rPfff99UYWY1cvqYpVscr0SkkuCiUCiEoaEhXHvttdB13dTRZWpqCul0GqIoGksFkUhk2Tu6ALWzMNdbWTxggwgmSREh3TXKFUoAmJubQ09PDyKRiG2KSDWi5abwQTFqVU92pVpurVSUrK7rmJycRDweR0tLS00aU1tFVNd1vPHGG9ixY0dBTl84HDZcudFodEXWrrzi1vMqXvp8aAHnOA6BQACBQKAguIisiw4PDxvBRdaOLrVcb1cUhblkHVj3gknCwos1cAacg19IFxGe54umiFQjmNVGf1YrmJlMBkNDQ64q9NSClbAwFxYWcPnyZUQiETQ0NKCjo8NWLGtRuIBER1rdbqlUCvF4HLOzsxgcHISiKKYAkJUSUYY3cWPJ+Xw+NDY2miLuSZAivd6uaVpB5aJKz61K5zEr660sHrABBNNt1KsoilBV1ZhU0+k0enp6kM1msW/fvpIiQqzESiqDVDtpV1pPlkT1Xbx4Efv37y9ZoadWLGeUbCaTQU9PDzKZjNEm7c0331xxdyVdMNwqopIkIRaLGSJKV5eptHUVY+1RTdyB3Xq73bkVDAYLKheVusZr6ZKl+5+uB9a9YALuJloieLquo6+vD1euXMHevXtdr3etZmpIuQE4dECPz+fD3r17TQWsl5tKg36KCS0dzUveD/neKn29WkOL6JYtWwDYV5fJ5XLGRFdfX89EdJ1Sy4o6dHARgQS6SZKEhYUFjI2NIZvNwufzmUSU7uhSy3FJkoRdu3ZVfRwvwQST2mZgYADz8/PYtWsXOjs7y7K2ql2HrIZyxNoa0DM4OLjiYlJLC1PXdaPRMClqb71r93Lqh1N1GTLR2Yko+VnuOqeM5WW5g5A4jkMoFEIoFEJbW5vxOF25aHp6Gul0GoIgGAKayWRqMi4W9LMO0TQNo6OjiMVi2LZtG06ePFlVekY1LGeUrVNAT7Xrp5UWUa/FGmY8HkdXVxfq6upwww03OArIahWorxS7ic5qLYyOjkKWZVMTZeIhYawNNE1bleIYJLiIdpcqimJEfGcyGaMoCR2hG4lEylrbZEE/axS7CZ1YJv39/WhtbUV7ezuam5srvrOq1sKstIMHUFwwS7XcqkUT6XIv+mosTE3TkM1m0dPTg3Q6baxTltpvrQuJk4jSTZRjsRgymQySySQikYjhznWzbsVYeeiYidVGFEUjuGhychLHjx83BRcRLw6J/qZduk7LBUww1wmxWAw9PT2or6/H9ddfj0AggP7+/lVL7SD7K4qydPKpCrhLvwB/+RdAKgZtxxHoJ/9PwFfY8cTutd223KrGwlxpwQSAmZkZjI2NYc+ePWhra3MlBOtBMO2wNlGur6/HwsICOn0VvFsAACAASURBVDo6IEkS4vE4xsfHjUoytDu3lh031gJe/P69khfqhF1wkTVwbWhoyFguoPuKNjQ0uI6Sff7553HPPfdAVVWcOnUK9957r+n5J598El/5ylcA5C3ehx9+GEePHnW1b63ZUIIZj8fR3d0NURRxzTXXmAoDV2shiqJo5HlWgkn0pocgfucvwF94HlCSAAcIAoBv/hEgAnqoDtqOa6B96A+gf+DjpvZgdKNqNy23BEGoeNzVpIeU85p0wYi6urqy8ynXq2Da4ZTPR1uipOMGHfwRjUZXrbLMSuCV3pw0XuyFWQqnwLVMJmOsi37nO9/Bj3/8Y+RyOXz1q1/FiRMncOzYMezfv7/g/aqqik9/+tN48cUX0dHRgePHj+OjH/0oDh06ZGyza9cuvPrqq2hsbMRzzz2HO++8E2fOnHG1b63ZEIKZTqeN8lP79++3vesRRbHinpRk/3Q6XfH+hmDOjEL8u1Pge98E5BTAIf/DL/3m0kkI0usQel4Hnr4bbYf+N4z+9l8hFqsvWaHHSi0szEr2cytgkiShq6sLoVAIe/bsqSggodTreW0iXQ7sRJSuLEOCP0RRNIot1NfXVyyiXrtB8aJgetHCrLRkJVkuaG1txX333Yf77rsPN910E2699Va88847+PGPf4zOzk78zd/8jWnfs2fPYu/evdi9ezcA4GMf+xieffZZk+jddNNNxt8nTpzA6Oio631rzYYQzEwmg/b2dtNkYaXanpQ1KW+XSkB4+u/Av0eJJbmeBJj/J1qoqQj3/Ad2TJ/D4I2fw9EP/j60eByZmRn4tm1b1nFXWk/WjdDKsozu7m6kUil0dnaivr7emNTLpRILc24ujcbG9e229Pv9aG5uNgV/0CI6MzNjiChtiVrTEOzwmkB5UZy8aGHW8nPieR633HILbr31VsdtxsbGsH37duP/jo4OnDlzxnH706dPG8crd99asCEEs6mpqaS7tRYu2WrXQIULvwT/8o+AdAYg8ztH/eYB0EuY+tJPIBvDgbf+Grn2BlzqCuD1Bx5AoKEBzZ2daDl4EC2dnWju7EQdFV4OVGdhCoJQ8R2p02tqmoahoSGMj49jz5492Lx5symfsppgITdomo4f/KAbX/vaWSiKgs7OZlx//RZce20b9u9vQkdH1FNCUGvsRJT0fpQkCbOzs0ilUkYaAi2i9ETLBLM0mqatSs3hYqx082i77ZzOm5dffhmnT5/Gr371q7L3rRXe+raWCTcfIqn0UynVWpiB5AICrz4D7socoFCTO3HJAktWJbAkqPRvWYLvpc9A9H8CifFxSKOjuNLXh6Gf/xxCMAheEBBqbsYd3/8+fKFQ1eOupYWp6zpmZmbQ29uLzZs348SJEzVrpVVqPzK5x2JZ3Hffefzrv/ZAkjIAVPT3j+G550YQCPAIBnls3RrC97//v2P7du9F/y2XG9Su9yNd43RoaAjJZNIkoqHF88sreE3AAW9amLWqI0vOxVKfeUdHB0ZGRoz/R0dHsc3GM/bOO+/g1KlTeO6554ybObf71pINIZhuWG0LMzQzhuCFXwHZXF78aMsSWHLFFkMHkI0jMPgtaDIHOZlBNh4Ht3hnzft8kMbG0PfTn6Lzv/23/GOrtIZJ70fWKYPBoG1h+2pfr5hgkrKJ77wTxx/90Zt4990YFMWP/KWhLb6eglRKQyqVA89nkMmsToEKN6yUKNjVOCW5fMQSTSQSOHfunMkSXYluG3Z41cL02phqVUc2m826umk6fvw4enp6MDAwgPb2djz99NN46qmnTNsMDw/jjjvuwBNPPIH9+/eXtW+t2RCC6dbCXM20kpZnHwMfmwcU3bxWSQunnVVJ/15E1OPgcjxUWQc4Ln8InoeWy0HNZND1/e8bgllq3GOvvw4hEEDzgQPwWbqXVBv0I8syent7IUkSOjs7S4agL4eFmctp+O53J/D5z/dibi4HXQ8D0BZ/VOonB0ABoEPTvBXQ4hXoXD5ZlnHx4kVcffXVhiU6MjKCZDJpFKunc/mWWzi8Kk5eszBrWUfWTS9MURTx0EMP4eabb4aqqvjkJz+Jw4cP45FHHgEA3H333fjCF76AWCyGT33qU8Y+586dc9x3OdkQgumG1bQwuXffgDgzAU5R8/M0kJ+jgbxQkvO32DxNntMAUQQ4bvFAmg6d4wBVzW/CcRh55RX0/PCH2HfHHSVFr++559D/4ovgeB71V12F5gMH0HroEJo7O6FXYZ0uLCzgjTfewO7du3Hw4EHX/SmraSBtJR5X8Zd/OYQnn5xAMikCCCH/wetYEk0FQBp5wQR0HUwwXUBcoKIoYtOmTaaCGaTovyRJGBsbQyKRMNIVSLGFurq6moqJF12y61nEyylacNttt+G2224zPXb33Xcbfz/66KN49NFHXe+7nDDBXKTaEnFVFfiOTcM3MwmoWmGwjw6zO9Y6RJuXzAsmvY1u+juXSmHgpz/F7t/93ZKdTvjFQgq6pmFhcBALg4Po/4//yA8xGsXN3/mOize4xMzMDLq6usDzvO06ZTFq2Ufz0qUs/uRPxvHrX0uQ5QAAP5ZEkghlDnYfMBNMdzgJlCAIaGhoMHkUSFWZeDxuiCiAAku00sl8PYtTLWHNo4uzIQTTa3eWVsR/+hKQSsA0OdNzsnV+LuWSFSyCaUFTFIy98goy8/PwNzcXdcmKRQp8a7LsWsASiQS6urrg9/tx+PBhDA8Pl31hViOY9H7PP5/GvffG0NOjQVVpF6yCJVesDLOZnydvYZY9hA1Hud+TU8sqYomSvo+6rpuaJ0ejUVfnkRcF04tjUhSlJmuYbl2ya40NIZiAh6u9jA1D53hwmlbc5UpTYjvRlxdMzmFTXdeRnprC+b/9W/zWP/5jUcHk/X6kZmYg+P0Qg0EIfr+hxlouV3Ldll6nJLVs0+n0shc8oKG/+0ceyeIrX0ljctIHXedR6ILVQLtg7fCqhem187vaG1We521FlK5v2tvbC13XEQ6HDXeuXZFwr7pkvWhh1qp5NLMwNwDVXljl7s+f+QXQ2wVOUZwjYUm+pdNzlt9uznddVTH3zjvIxeNFJ1oxEEB2YQHa4vosx3EQgkGIwSC4QACp6WmASh4maJqGkZERjI6OYteuXaZ1ylpF17qF4zioqo7PfU7DN7+pIx4PIJ+jo2FJLBXkxTMN8xdR+F16VTAB73hTlkugeJ43LEuCVURJkXDaEs3lcp6z5rxUfJ2gqmpN2sYxl+wGgESMVnqHRSwZtxOFtLCA5Lk3sE13lzUCwBTcY+u21W3WMO0Oo+tYuHQJwz/6EXDggON2gt9vej+6rkNJp6Gk09A1DT3f/S4OXH+9aZ+ZmRn09PSgra3Ndp2yFpZiOSSTPL74xVY895wf6bSI/IdFol/pv+lKT3YfYP71VZX5ZEuxkhadk4iSIuEzMzOYm5uDoihIpVImd66b8pHLhVddsisd9LOW2DCC6WayJZGulQom2d+p3Q2BtKdKJhK46ZcvQMiklyJhnXAZIQsAAhFMJ58sAE3XoWQymHzpJXBXXeV4aMHvB5wuao6DkskY/5Kem6Io4tixY455WNX0wyxXMEdHgc98Ziteey2EXI7O1yHfMbEuU2R09EgLjqfr5hgqhj2r7QLled7o4bh161ZMTk4im82ipaXF6LQxODgIRVEQDodN7dBWSkR1XfecYNbSJdvR0VGDEXmLDSOYbqhVaomTYFpbbh1u3wY+GLbd1oRFEEuPo7SFCeTdsrO/+Q2a/ut/ddxGCAQATYOazYLj+fwkSH4DUDMZ5HI59Pb2YmFhAQcOHDAls9uxEiXuAOD9LuAPP+3D+Td9UDTOHHls/CPA9QcLllbiltUWTCtEnOw6bdDtqoiIhkIhkyVa6iZ4vbAaaSVrCSaYFMtVvMCp5ZZw+n+BGxkEdLW0hWkczPLbIc1EcLOOqevIzs4i9cwzwH//77bbCIEAsGjZ6dR74wCA4zB74QJ++W//hgMf+AA6OzuXtT9lOUJ75k0Of/hZEZcvc1BF5A1LEgBL1oQrrDNhN3avCcRq47XPw8n96dSuKp1OQ5IkzM/PY3h42Oj5SAS0vr5+XYooE8zibBjBXIlqP3b7x2IxdHd3o6mpqaDllh4K5U1BlXPv57MNe7UbS1GP7NKuqgqlu9vxeesaprHfom8yPTqKq/x+tLe3l3ilJSqdSN0K7Su/4XDX/y1idJQSS2sgbL5UrPUVXI1DVZmFWQqvCWY57k+O4xAOhxEOh7F582Zj/0wmA0mSsLCwgJGRkQIRjUajNQmYWU3YGmZxNoxguqGWFqYkSeju7oYgCDh69CjClrJySCbA/fiHgJIvt1Y19CG0fGqJ0/xPG6mqokDp6cHIk09i+//4HwXbCn5/vhZtMaGqoiRgOZSagDUN+NnrHE59TsTsPAdtqSSsOXtEhlttpF8dAFnDNH8W5QZ7LRdeSivxwudBU21nELrnY9tixx+riI6OjkKWZQQCgQIR9dJnUYxarmHS1Z3WC0wwKWphYabTabz33ntIJpNG3qEtPj/0wT5Als1dSNxAImSLzI+lznlTgK2qIjUwYLudIZiAWTTJBKBp0GS51IiXHVUFnnmVwz1fERFLctB9WBJL8nmRrJEq0DQN8/MLkOXNnnTJeWVi9qJg1jrAxklEs9msUYR+fHwc2WwWfr9/TViitfreJEkqWRt6LbJhBHO5XbKqqmJ+fh6JRAIHDx7E4cOHi76mfmUe4EuU5DE2dvGcRUBLBfqZDpnLYfT0aWy7/XZELcWLBb8fmp3lsviYkkrh/AMPYOctt8C/SpU9VBX49vM8Pv9NEbEsALrSHbEsFeQtSxpbn7V9Ogn9dzwu4b333jOCQ2RZxtzcHBoaGlY1TcFLbATBtIPjOASDQQSDQbS2thqPExGNx+OYnJxEOp1GNptFX1+fIaLBoDcaltdiDIlEApFIpAaj8RYbRjDdIIoiMlSahBvogJ5IJIKdO3ca6x7FUJ9+EvzYGKCp+W+hlDfNalU6lMUj//tcGD8m12wqhczQkEkwY7EY3r98ueTQMrEYlHR6VQRT14F//Dce939PwHwGeWudWJZ0cA/9JkrOB84bcBywZctWXHfdTiPC8t1330UsFsPQ0JApYb6+vt626sxGwEvuYWD1BTwQCCAQCKClpQVAvgLWe++9h4aGBkiShMnJSWQyGfh8PpMlGgqFPCGi5aLruueqGNWCDXMlL4eFOTs7i56eHiOgJxaLIZlMlt4RABcKQ+f4pUm9kvnFWriAUsByDR01kcDAF7+I1ttuQyqVwuXLl8FxHA5efTW6/X5kHcegQ9f1VXPLfvEHPP7ppyLmFeSbjRCrkgimdYnY9jQoZVWaIWklJMLS7/djz549EEXRlDBPl26ziuhyTCZeEykvTfReKxJA1lRbWloMEQXyQkrcudPT00in0xBF0SSi4XDYU5+tFX1xTliPbBjBdINbwXQK6HHbE1PPZKANDkIAoIMDV02ErMPjPr+7uBZjV02FPDODS7/8Ja6IIg4cOICmpiZc6e/PX5ycfSSvDkBJJjF/6RIiK5yo/Jc/EPDozwVcUZB3w9IpI7Rw2lFUOJ0+Oc42D5OO3rUmzANLpdvi8bipiDidLL8SPSFXktW26Kx4rUiAU/qG3+9Hc3MzmpubjcdyuZzhzp2ZmVk2EdU0rabfmZe+/1rBBJNCEISigmlU6HEI6HFtoSoKlB8/AzGdBkRLh5JS51g5Ltkylkd1HchOTkD76U9x4stfNk52IRg0mlA76bUsSXj/oYew/cMfLv2Ci1QTWSorwH3/LuAbvxSQVJEXS7osLCneQ3vXK7p2SbkkM+XWXLAr3WbXzorjOFOeXzgcLnuS98ok5TXBrLUYVEs5Fq/P50NTUxOampqMx4iISpKE2dlZpFIpCIJQIKLlnD+1ysEkQU7rkQ0jmNW4ZOkKPXv27HEM6HFrYSIczhcDIP+7ccnaBffY1ZJd/LuYS9Za84D8z2kaxKEhaJIEYTGHyikP04qadXTa2kKq9pR7gaYVAX/5HI/HzghIcLC3LMlP2RR7n3nxLJZWUg527axUVTUmwaGhISSTSdMkSETUSxO/E14UzLVgYbrFTkQVRbE9f+ieonV1dY6fQy1zMNdjay9gAwmmG6yC6VShx+3+TmiXLkHX9MqzLx2sSvr/UuXx7A6hqxqkX76K1HvvIXrTTQBK1JIl+1aQWkKq9pRzgV7JAN+4vBsvjYv5MulOlmV52l02di7ZWiAIAjZt2mTyXNCT4MDAAFKplMkdV19f78nAECaYxVmO8YiiiMbGRlNpSkVRjJ6iw8PDSKVSJk8GLaKstVdpmGBS0BaiNaDHTbqAWwsz9/OfQ5+ahK5pWFzIrB6L1VmscEGx3bXkFUi/eHlJMEnSdZHJT1NVzL/7LoaffRZXFalLS1NuPdm5NPDnLwt4YW4LMgKAIMzWJFmzdFtDoeDtWB8oHfRDs1xBDnaToN2als/nA8/z8Pl8SKfTq56i4DXB9Np4auX+LIUoirY3YURER0ZGkEwmwXEc/H4/FEVBPB6vak19vbb2AjaQYLqtcaqqKt58802IomhfoacIbi1MLhhcjJBdXBksZ651WYhddJHiaXcoNati/qlvYtu9nwNQhks2l0NiaKjkdoRyBHMuA3zmFQHP9gh5sQSWzlw6z9KuKIFd85EKq/wAcAz6WUns3HGyLBtuuJ6eHqTTaQQCASOoqL6+fkWT5b0mUBvBwnSLnYiqqorx8XHMzs4aa+oADHdufX096urqXIl8PB5nLtn1QLG1JhLQk8lkcM0111RU1smNCOialheqci+WYhGydrVkfe5qIhSgAsrsNBJn/wWRG38PHMeBd7OAr+vQcznXL+O2VVcsA9z1qogXxnjIfiw1FyERsCLMlmXpugMOuP+wVtLCdIvf70ckEkEgEMBVi+3astks4vE4JEnC2NgYstmsqQvHchYQZ4JZnJWyMN0iCAKCwSA2bdqEXbt2ASgMTEsmk0Z0NzmH7FKkmEt2HaMoCgYHBzE9PY09e/ZgYWGh4hqIriYIWUby7/8evnQaOq8vBca6Cfpx6ExiF8Xjqom0w8tqCQ3zP/ocIjf+HoDFdUyHg5H91VwOs2+8AXlhAX4XJbHc3FwMJoD/+YaIn83wUAQsVeahy93l4FzurmaRsaUtzNUWTAJ9DgYCAbS2thoVZ+jap1euXDEKiBMRrWU/SK8JptfG4zUBB1DQC9guME3TNMOdS6dIkXXQ4eFhzM7OuiqL9/zzz+Oee+6Bqqo4deoU7r33XtPzXV1d+MQnPoHz58/jS1/6Ej772c8az+3cuRPRaBSCIEAURZw7d64Gn0BpNpRg0hObU0DPwMDAsp7MXDCYty41famLBi18dte0UzSs3f+LuO2JaftSOUB+fQJzE3+Exq1/l3fLOr9Ufj9dx8xvfgOptxfN119f8nVK9bZ8ewH4kwsizszxUEgRdWsUrIy8K9b2BWCvfXB4rAzWaj9Mp9qn6XQa8Xi8oKkyEdBoNFp2MIjX8h69JlDVFoNfDtxYvTzP24poMplEb28vvve97+Gdd95BJpPB+++/j+uuuw7XX389PvjBD5puWFRVxac//Wm8+OKL6OjowPHjx/HRj34Uhw4dMrZpamrCgw8+iGeeecZ2LC+//LKp6MNK4K1vbIUoFtBTqgl0TVhcS9IrqfDjlH9peb6UYGqwN1ANi7FbRbrrx/C13wgxGHQ1JJRR8aeYhfnKHIfPvi/gYnKx44g1EpZYlrQHuJQI1si48LqFWS50KytrU2USVNTf32+U/KNF1EsuxVJ4zcL0mksWyI+pEu8CyTM+duwYvvWtb+GBBx7Azp07cezYMZw/fx4/+clP8MEPftC0z9mzZ7F3717s3r0bAPCxj30Mzz77rEkw29ra0NbWhp/85CfVvbEasqEEU5IkdHV1FQ3oqVYwieXkdDeb+fnPoS2uBQC1CZC1O5BQhoVZoNs6oEuA+v8tYP7mh4BA8QVRHQB0HWomAzkWc/WaToL5vRkOf9nDYULloAW5wqo9xSxL5wI9FWK3+KmXXbhgrUE3VaarFRERpUv+0UEhdGSl1wQK8E5RB8B7Fi9Q2+bRTU1NOHLkCI4cOWK7zdjYGLZv327839HRgTNnzrh+DY7j8JGPfAQcx+Guu+7CnXfeWfW43bChBHNhYQF79uwpukZZq56YjsnBw8NQZ2cBVc8HsBBKWZv0c5rlfxtTsVKX7NJAAe0VGbo8DyUYLzG4xV2SSbz/V3+F9v/yX0puaw36kTXga6My/n4mgIRfhE6CeaypIxlUZy1WVBLPjFXo17KF6Ra65B+BXs+iqxVFIhHkcjlEIhFPCoMX8KKFWavCBW7SSuyul3JuaH79619j27ZtmJ6exoc//GF0dnbiQx/6UNljLZcNJZjbt28vmSdZqjxeKYjgOrk2uFDIiJC1WnUV4eCiFUWAczFPORpLGoBpDfyPEuBCCvQSCY7EytRcRsrSFuZ4Kov/OZzBC3oTMkHB7HqlXbEpV4fOU5GL1mnRc+n5fKUfy6MeEcyVHgO9ntXe3g4gLwSJRAJDQ0OYm5vD7OyskShP3Ll1dXWesvZWAy/eSKxk4YKOjg6MjIwY/4+OjmLbtm2uX4Ns29bWhttvvx1nz55lgrka1MrCdIILh6GTsnil1iNprBZoiX1EcXHqLxWtY3NYolWcDHA/y0IIcu7q0uo6dFWFrmlLTacdIDmvzwyP4wtKA3ojTZDVxe4tJE2E/ptU76k4baR2qKp3fbKrLUSCIKChocEo49fW1mYkysfjcVPJNno9dK2U/KsVXrQwa+mSLRUle/z4cfT09GBgYADt7e14+umn8dRTT7k6fjKZhKZpiEajSCaTeOGFF/D5z3++6nG7YUMJ5nI3kXaz/8I//AO0bBY69LwwlSOaBKdisPQ4FtMwiullsYBbHYCeA3y/yEA8Wjzox9hH15EZG0Pf3/899v7pnxbdNp3J4CFVxb+1bccM50NOlQGNB1QBUDlA5/PvL8eZ00bcFOQpJqDW53SnDZ0fd2thZjIKVFVHXd3GaypNr2HaJcrTxcPpDhx0oYXVrla0nHjVwqyVYJayMEVRxEMPPYSbb74Zqqrik5/8JA4fPoxHHnkEAHD33XdjcnISN9xwA+LxOHiex9e//nVcvHgRs7OzuP322wHk3cgf//jHccstt1Q9bjdsKMF0w3JbmFoiAV2WK3PB2u3jsJZZqnBBqUORB7gZFb6082DtXj49Ouq4fTqdxk+HevHYtijebtiEtKaC1zT4NB66BkAToGkcNE2AnvYjn4BpA+09tUshqWqeLRLgpAOqWvrLe+edWfzt3/4G589P4KqrIrj22s04cWIrDh9uxf79TfD7vWVd1JpSQT9O1YpIyb+pqSlkMhn4/X6TOzdASjWucbxoYVrzMCvFbeGC2267Dbfddpvpsbvvvtv4e8uWLRi1mUvq6+vx9ttvVz3OSthQgunWwkynnTLhS1NKcIlLFppqXkB0G/Tj5A3UzT9ui6+XrLQnAcGLpauZG8aurkPPZAqeV1UVb4704/FQBq8f3oSYwIPT0wjpPDSdg6bxUDUBuipAUQTksnlD00S182RN5tnC5ri0hZlIKHj44cv4xjfex/R0vrzYzEwcb721gMcfv4xgkEddnYA//MNr8ZnPHK/FgDxJJVGydr0gs9msIaLj4+PIZDIIBoMFIrrW8KKFWasxVZqeshbYUILphmVfw6yrM/42NLLSXMwiz5UTJVss4BYa4L+iFRUbUwCvomD2pZcQf+st1B87BlXXcHFuAo/nxnFudx2uiCJkpBAEB40ToHEcNAjQBAGKzkPR/FDUYOGRS1mPboJ8alD5x87CJMJw4cIC/uzP3sf587PI5UQAEZCIJU1ToWkKcjkFqZSM8XGpksGsGWqVVhIIBBAIBIwEdV3XDRFdWFjAyMgIZFlGKBQyrYnSaWFeCMiyUiySfjWp9jvz4mddS5hgWljONczEv/875K4u8KoKrUCZysCFeWjkYZYI+impvTogJnRwuvuh5uILGBjrxtvXhPGqMoHxNgEpQYAKCUHw8EOACh4qBGjgoUCEAhE6F4RifRWr67Ucqp6v7Q+gKOYPPJvV8PWvz+Cb35xCPJ6FrtfBnBOjYqnvWNL1q2uajiefvITTpy/gyJFWnDzZjmuucXbpemmyWq48TI7jEAwGEQwGC0r+xeNxzM/PY2hoyKhW5OWo3JUQTFJkg+dX9v178fOuBRtKMFci6EcQBGSdmikLAtRkEry1aEEFFX+UNCAvALkFgPcBwb2Ar2PpeBwHlFoicRsMJBQ3MM376jrSehY/UH+DvjAgw48ARIjwQTGEUoQKHjn4kYMPOnio1KtyRUdnoZLrsuwC7ObHaQvzrbcy+MxnNLz7bhK5XAiAH0uhvfRPBvmKC3mKldfTNB2vvz6Lv/7rczh/fhK5nIy33lrAd7/bA7+fQ12diD17ovid37kKf/EXN5V4s6vDSoo3XfJv8+bNxuunUilIkoRYLIZkMomzZ8+irq7OVGjBa+Xpaoksq/inf3oLTzzxHvbvb8QNN2zB4cOtOHCgCTt2NBSIaC2+s1qtg3qV9fvOKkQURVc9LYvtn0zaWxFGHVkK0znqsnDB9CgwNwZoWYBbbKkpXMj/9glAYBMQPAoIfOkoWTtjtRLBJPtwAMREFld/60XIH21HKhBBBkHI8ENGEDJ8i//7oEIsrXel1jHt/ncjogXbuA+3zVe90fHYY0k8+GAck5N1MDfkJD+kLFHh+eA0Nw0NpXH//T344Q9HkExmAdQBCEHXVSiKAkXRkEopmJ2dw5UrcoFgeunOfjVdjnS1oqamJqTTaRw5csQQUbpaEV3yz677xloklVLw5S+fwz//83vIZnPo70/hxRfH4PdzCIcFNDQEcOhQE7785f+E7dvra+YRkCRp3bb2AphgkNUMPgAAIABJREFUFlALC9Npf76uDhAEW6Eq/KcQTQMmJ4HxMUDN5adwAfka7jz5WwV8MSD48/LH7rSWyQPgXN58agCg66i7NAsxJUMM5B2vxLbkjCrqdq+sm2WrFqXurO5c1/s6bzg3p+PUKRk/+5mKTCaI/GVkLUlEGnTaF3KwWpiplIYnnpjBgw8OY2wsBU0LIm+t0kKsLB4vA+eq8/ZMTSWxaVMAgcDKXPJeKo1HglnoakV0yT/SwmpiYgKSlF9bjkQiJhH14nqjE/F4Dn/+52/hX/5lELIcAJA3AlRVhyyrSCQ0zMwkEYtlkE7nz6NaRe2u5+bRwAYTTDcXcC0q/ThZqFeeeQZ6NpuPJAUMndDhTpDm4sDgolgCS8YUEUwOQP7yyB+znC+3WJqJ4GJsppsAPf9+hJxCPVtIieq09rhNHSkryMfthhw0bTsef/xqxON+qCppzmldq5RR6u6HtjBffz2D++6bxhtvSJBl0pqFHItYqrnF45bn/ZicTOHBB9/Gv/zL+9A0Dbt2RXHs2Gb81m/l10N3794EQai9GKyV9VRSOJy2iug+kKOjo0gmk0a1IuLODYfDnhTR+fkc/viP38Vzz00ZYmnuXpA/n3Q9nwkQCuVniVrmYDILcx1RqoyZ28bGThSzUFOXLkFNJMw2lp2xZXNtZxRgaBZILIolD7NQ8ij8Moud/rQFWapuDXkNt3A64J9O4qpvvoH3P5dPKC7bS8qh8C6iUovT1eDd+Hqvg6pej/n5EJY+ObvAnmLlAZeUfGFBx9e+lsTjjycwP89jKaqWHIv8Tdy65k4wxe7/FhZy+MY3evHYYz2YmJCg6z4ACmIxCefOXcFjj3UhEODQ1OTH6dO34OTJ9iJjLh8vWphusesDqaqqUWiBrlZkFVE373m5biZisRxOnbqEX/xiAblcEEulssi5Ss6pfNqXIHAIh/PpH7WqI7uwsMAsTIZ77NJKNE3D4OAgrsgyotQFpcN5LctKLAnEUvmpmLP5EbFkWZKfUl9u0XQS6m/Su9kJO8HlVA0NFyYKXtHsGS3839i/onVI6jEn/atoDg8A/A2AvhP5qvDkQPQkrCNf7LZ0zqquA+PjUdx6q4JLlwSoagSFFmVpt64d6bSOb397Al//+iAGBxPQNB+AepjduipUVUEqlQHPy0il3B/fLWtZMO0QBKGgWpGiKIaIDgwMIJVKQRRFU45oKBQq+ByWo2jB5KSM3//9QZw9m4WiBGFu8UPf3C3lmAsCZ7IwV7JowVqFCWaNoS1MXdcxNTWFvr4+bNu2Da07diDDcYWiRKucA73zQEZdMkCtxW0Em91LpQ4X02paBN1MNQXH0gE+mys/E6TifMllQogCwSOA3pgv3WctDO86wpm8KR807QReeukIFIWHOUCIWADECiin2jzw0ks5PPxwCt3d8mIeaBRLImyN2E0ZY3KbcqCqGjRNh89XerL3kmAu11hEUURjYyMaGxuNx0jJv3g8junpaaTTafh8PlOOKFlPrRVDQzn8/u9P4MIFBapKLEsimMTKVJD/3slzOnieWxaXLBPMdYTbzhKVXmTEwozH47h8+TJCoRBuuOEGiIqCCaoRM7Eu3dSSHYnnXbHK4jZWTSEradZDVPLl2s3/pSxM6/5A3sJsem0EW164jP6PHLWtZAeH/wuP5rCTFbdWZNHnqSeDrUB0H4AgoHNmDxfRnhws8TfFDr4FwHUA2qCYSv4tTWJLVmVpSxXIn6eXLqn43Ofi+NWvBGSzYeRXsmmLla5gn4L1FqjUea5pOs6dm8ZXv3oWb745gT17GnDddfl10KuvbsXOnfYpCl4RzJWsquNU8i8ej0OSJExMTCCdTiOXy6G/v9+wRiutVnT5soI/+IN5vPceD00Lw7xEQP9dWL0sGBSM74gF/bhjwwmmG0q16CqGLMtIp9Po6upCZ2encfKkLl3C/I9+BL9SkJpfFE0H5jL5fpHWcA864IdQjkuWbE9+27lWjShZl+OlA394WUVoPG7Z1yrJhRLNcYuvZ6ewpVytVc3RZGceaG4HGrYCIMXgUdhFpdRypYEI+A4A/H5AWxRf+g5HJ/HOQOkV5SVyuRDuvVfDd7+rQJL8i8egRZJeD03AugYKLH7WRT6zs2fn8Hd/dxGvvjqOVCoDQMTs7ALOnJnHP//zJQQCQEODDzffvAMPPvhhY78VFUx1DmLiBfC518HxcWiB3cgF7gLEfGGD1S5D5/f70dLSYlQrSiQShliSXqLZbBahUMjkzi3VxP7tt1XcdVcaXV0CNC0As/uVtjKTWLrOljo2BINLM0St8icTiQR27NhR9XG8ChNMGyoRTE3TMDQ0hPHxcYiiiOPHj5smDD4cBqiAIuKKdQz+oRiQ8oJJu2MJKpy/RBHu9KNUKmi5QT+GlaloEBPEDUStTy5uZb9+WcbtRKk4HetjbiJseR7Y0Q5E6wHwhU2sdeRFMm13DJsBCQEgejA/eetCoYVKjMqyArNF6PpRDA3diEceqYOu2xVKIEKZQqnIWjuX7JtvSvja1wbxyiuzSCZl5PNBg8i/+bz7WFVVpFI5pNMZvPvurGn/FRFMJQ7/5D/CF/sWOH0G4PX8SS8AAeF+QAR0HxD0t8IX+l1A/X8BIby8Y3KBruvw+XxobW0tqFYkSRLm5+cxPDyMXC5XIKJkTjpzRsenPqWjry8ATSPRC7RlSQehAXY3peHw0szBXLLu2HCCWetqP7quY3p6Gn19fdiyZQtOnDiBM2fOFLwOKVpAn7b5nEUqMNZGKwYlIEcZHVZ7zO6+mTzv9sstllKio3i0LY3VNuKzCg5/7VX03XEcV7a1OryaVUgt29RiTdPt/gEe6NwKhAL5GYUWSTpgVSl1zMVB1zUALVcBfBDQOLNI0nE9ZTXGbgNwBNC3LK5TAkvfNH1g92ugtGBeuJDF/ffP4JVXriCRUJCP3CVWK229koHnv3VBWFmXLJcZR6DnjyFKrwLI2udYCQAnAGJwBltD3wbmvg29zo9s4P+Awn8J4FZnYrcTJ7paUVtbG4D8Z5hOpxGPxxGLxTA4OAhFUXDp0hbcf/8ujI4GqZslchdGu2ETKFxwh/GbrF+SMZWyaN3ABHMD4lYwJUlCV1cXgsEgrrvuOgTpNUrLhEEsTNitoVrPZWqeSSp5wSzlUCK7W4N1nKYsu+2dhuTGwnSyDzlNh5AtJwpTt7E4F1/djYVYKfUCcF1jvpGoNWOEzEUkFdLNa29tBlqaAU60t1LJzT8x/rjFx50OzvmA0BFA2AFo/ryb2HpMw7Xr/i6DnKOXLqn40pdSeOmlFJJJDvnIWqtQElM4BVqMOa7QSnUSzIWFLB577B0EAgJuuGELOjubUV9f5vqdHEPg/Xsgzr8C6LI5t0qw/E1iYMhYdRk+fAe5wA+han8FQfm/wLm+JawNbl3EHMchHA4jHA5jy5YtAID/+A8e998fwOiosBj/QKZwWhhlmG+WCpdAOA6mPq3MwnTHhhPMWliYsiyjp6cHyWQSBw4cKOguTgJ/6DWBdFcXdOqYJoFx8EImFaA/Dqi6O8G0XhaVpJXY4Vas7Z4QkzIaL04itmsbCmW1yKsW+5oqEUs79y15bCuA64KAj1u6WadFk6xXunE6iBywpxVoCgPg7a1UEqjqLq4HCDcCDTsALmKO1LWrl+CqrsHS568om/DVr7bg9dd5pFIk0Z22IknkLvFDW63W/LFE0Vry0SyYyWQO3/52Hx5++G2Mji5A11X4fEAwyKOjI4xrr23DyZPb8J//8w5s3158wg1c/CLEyVfzb9gaMk6G4af+tnz3OgeAS0EN3IeM+ApCmUcgYBNWikrF6UfPCrj3L4KYHOeodDQ6Th5YuhuzC98z/08LZq3yMCVJMqXerDc2nGC6walaj6ZpGB4extjYGHbv3o1Dhw7ZCjARXFow53/2M+RmZiBoeZuOXrt0kg0eedEsFQbitH850a30cawRt6WOU0xsfYksrn3oVfT+7jGQhJolC5L8bxHQcsSylDFVLByXB7AbwDE+n4NjFTYy98hYErdir9XAAwcbgHox74KlBY0WYLcuWJ4Htm0H6psAXSwcF/mfaFlZ1AM4gVTqAH7+c+KKoxdXaauSrIM6v3nrdUD3B33iiXE88kg/hoYWoGkCSF6oLKuQZQUXL2Zw8eIAfvCDXnzxizfh7ruPOb6OOPpjiEM/BJTFnqu0dUl+06myRde5dajiGcyH70B9+hH49f2Or1tLKmnt9eQPRPz1FwOYmeOg+5B/n7rNDwBYr6cCF1b+JxIxW5gsD7M0TDBtsFqYuq5jZmYGvb292Lx5M06cOFH0bsyueIEYiZjWMHWgsGiB5abwUjwf7FOKwtWJPOUG61j3J/+Td8rZPO+0rzEeTQcvuzHNbO6IK0kPceuNFAAcA3AYS0ms1iBDcqPuhnYA1/iBkMWqpMVSBimyUnqM9X5gdwsQpKJqyZonrWvl1TUAuAAQPIB8EYbgkmvXcOnyWMrgzSG/DkYP1v4Dsa5hJhLAww/P4dFHpzA8nISm8TAXUKAt2HwlI11XbduWGUPPzMD//gPgMvGlB2krkvw4HcJ085T/RwOg8uOYCt+F5vRXEdZucHz9WqFpWlnW3De+68P/8w9+zCW4vIuZPrfo88I4D4r5jpYEM5mcx7lz5xCNRpFIJJBdLNtZzdozE8x1hluXrCznQ/ATiQS6urrg9/sL1imL7W916fKhkNGpxK0rNK3kBbOcpXj6ZtONhelkVdJUE3fDqTqaLk6h/bU+9N90yFX+JcfpyJfFswT/2P1tHajTwem/fQBuALAHhWJJl24tdbNCjnkQwNUcEOAKhZduWuJG2AQAu4LAjob8+UIHPVqDhdy31gR4AYjuBOo68qktVguYWKxEw0zrqcVc6MQlm992bk7F6dMpfOMbjZidTULTQih089ILwkuKz3GccRw7fBf/F/j5y4CqLt29WQWTDmx3OJTd8oXKXcF46F60Ze5DvfpbjmOoBeWkudx/2od/+JYfUpbLp9dazwVyY1dwT1poUdIpJRwH7N9/Fa699lpIkoQrV65gYmICAwMDRmlAEp1rV63IiVwuV3FO6VpgwwmmG0RRhCRJuHjxIiRJQmdnZ8E6ZTHsLEwimDrHmU1LO68JGcfiNeU+M88yDhfblF7pcB8lS6DFlwPgk7LY/PYo+m86ZLyCxYFn/GW+Lku4aJ0szGKPBQF8EMB25N+YNSCHiIbTWqD1+B8A0InChiW0uGVhlwJZSB2A6wSgdXHWdxqb2/VUQvMmoGUPIIaWLEprcSEy6SbLPPYishzC5z8PfO97KmZmeGhaM5aqOtAfKnkDKZjvIPLftVMheH7uIsSBnwCZ9NJ3wC0emvzPY+nu0s38zpkXBzROwnDwy2jP/Dka1d92cYDKUFW1ZMqaogJ/9XAA3/qRiJTKASEUei7IjZitWDpd1UuP19f7jWpFPp8PBw8ehM/nM6oVSZKEvr4+pNNpiKJo1NeNRqMIBoO2bngvFd1fDphgWtA0DbOzs5iYmMDBgwdx8ODBsl0UVgtTUxRkRkdNamDy/jncwPOcewuRPiZ5vNw1TLtjk5t4t8exu1w4RUVwPr9wV0z7Sr5GMSvTjXs2DOC3ALQg/6bowEIyGdlOQA7H/h0AO2FeT6KtNhIs5IYOAMcBRPjCY9FWYDnBQhEB2NsO1AXzA6atVXqMxNArWyh1APXQtN/Ga6914he/CCA/s9tF1pLB2/mQF9eyuULXLkG8/BS4hRFAXbx9tAv2sdNazvy7+JICB5VLoS/4AHZmBLSoJ4tsXTmlLMx4CvizfwrgX1/xQeZgFku7QK/84G0oWNw0/uc4IBpd8l3RgUhO1YpIyb/JyUlkMhn4/X7DCgVgbO9mvnz++edxzz33QFVVnDp1Cvfee6/p+a6uLnziE5/A+fPn8aUvfQmf/exnXe+7nGw4wSz2Zc7MzKCnpwf19fVobW3Ftm3bKnoNOwtz8umnwSWTS629XCCSyaDEDnb3k0BpoSvmhqWPWWxZqNixyP9CTsX1D7+Ktz92EyZ3X+XCu2pxxbpV62LbRQHcBKAZS5aJ3QTkxm3qB/DbAK7CkljSrjJ6fdEN1wI48v+z9+5Bchz3necnq6q7p7vnBQwwwAADAhiCeBEPviBivZJ27T1bEuKCsXR4Ixg+W6Hg2RJlaa1br2PPG3t7ew5v3Gn3rDhLRzm4q5Apyz6fbJ/CYniX0pq2tPJDEiiRFAgQ7zcweAPznulXVd4fVdWdlZX16JkRteTgFzHTXVWZv8quysxv/r75y1/iU2763JRatlwesEH5dhRhtOLvIq6Cox66NtVaTaZgcQag+AR4G8F1aHnB6E7a+LUl7IzDUcgcZqSP3kP3tgWQE+eZPfYKA40FIlN/KmCqUTqyRmUx8IxmaIkFTvV8Dq9mM+y+J0HZ4iXNS/b6hOBXPl/i2z90aFl06oQ+gAop/tgrSprwkZE/f4PtDmBKKVNBvFgsMjQ0xNDQUPtcvV5vg+hv/dZv8cYbb7CwsMBv/uZv8sQTT/DEE0+wbt064+//xCc+wSuvvMLo6CgHDhzgqaeeYvfu3e00q1ev5nOf+xxf+9rXus77o5T/9jZ0+zHI7Owsr732GteuXeOxxx5j69ateN5iidC4hWk5DqJQ8OlYXfTbKHXcUSzMvM42qiwmBqx5PLrIAOzKOSEldsT5R2pes3S+B787Eoh9KabyauAD+AAXBqxR/8pkR6oPpQf4GXwaNgzbGv6VFZ15aGQL+AfB3+pAXzn4VP8KKfp03ZuAD1mwrwirLD/uQBX/M/zrC8q7mNZf6YeRA7D+p2BgC/QXYED4/jzhvcLVKe0fCXlm7oUQMUr2L/9ylv/zf/pPlGYuYEsZBXx1ILBo6TzYTo30QfPNnt/luv36UpQbJcnCPHXd4mc/08M3Tzu0ivjvvkqnXoT1S9+D3QiaOm0VbdnqTiWQzyrUpVQqsWbNGsbGxnjxxRf55je/yaZNm9i3bx/f+c53+PCHP8y3vvWtWL5XX32Vbdu2MTY2RrFY5JlnnuGll16KpBkeHubAgQMx6jpP3h+lrGgLs9FocO7cOaanp9mxY0d7/VC9Xl/yJtL1enQ0bfX0RJb4gU/Fto9T5jDzit6GugVME4ETSjdFMbXf0sQc+/70+4z/660JqZcw95Fmha7DpzrDDSVUGjbseGsp+VWp0qF0Q8tftwjDKbssqeAD5aimS52jClnMPFIOyraFztysaQ60SbeboPjlW9ULI6NQrPhrQXVHIZV9DacoMyX6zoXoOA+98kqTf//vp3n99QXe+th/oMdrdCIsaZZiauXU0xJvJyb3GICWqPN6+T+wb+E5NrnJS126Fd3ClBL+/KjNr/2/JW5PWsiQgtUpdA8za5EJmDr/I7EsKJc7YLQcc4+zs7OsWbOGp59+mqeffjox3fj4OJs2bWofj46Ocvjw4Vz3WEre5ZAVB5jgj/CuXLnC1atX2bp1Kzt37owAaTeh8UxiomTt3l5czcmgXUUT6qqdYw5Tl7B5QPfOOiZdoYSMVzfNKjKf2nB58Fsn4F8npzdZnAkJ8z2YB/A3Bwm93NURi+o0kaYrvNaPD3Aq8OZ1FtL1r8Z3FhomPu+ply+PbMEfFPQr+nSwbBKnX7PGKjawrgBj66CnAFJEgVKfB63jr0LJFDN573nwgx8M8JnPwNGjgmazwofG/obB4ixCevGQhOou6mjXsnn/REpWPdekwffKX6JZazHWOpD1w3KJamE2Xfjfv1nkhb8pMNcUnWUjuvOYS5eDkKShr3/OB0y/+18uR528S0pM98tr4S4l73LIigNMKSWvvfYag4ODPPnkk8bFupZlLSslK13XD42n7YUJGmhqFwsBMISWaDcOQKqFmQV0Wb801JXnvuq58E8AwpOU785RnKtRq/pLc9LnL6WZjs0Llg/izw32Eh1kq2CS17IcwPesHaDzMHVLNW+UnfX4YDmo6NE7x7yBCGz8edkddGg6XV9YNkn6FKL6Ei1gM7Dbgb6CeQ5UBcrQUSqbdU244AB7qdf389nPDuE3PYklGjy+/jT9hTksNW6CWqk9ci0liUlKJ6uXsEWT7/R8hflakz2tn8h5g2QJAfPMPcGv/ZcS37ng0LTxWQd97jqsDyrTkEUHxazL+HnLElQqhXZ5livKT19fX2a60dFRrly50j6+evVqbn+RpeRdDllxgCmE4PHHH0+d4F7qiEW3MG/9+Z8zfeQIBc3qVKnZ2Em6o2RNTjdLiQGrp+lmiYpJl5Cw9uwtDvxv/x9nXYvpPWNM7tlGc88uqCQBqKZJB0udbgu/78S3LMvEKdM0astklQwA78e33lRnIRWc8joLPQC8B38OUX1YenCDPORGH/Df4Yf1U8ule8A2c5YtlK34A43V+BalGlFI78QXWML8ofR3DunZBvYD4FbAtfBCYPQEo303+dijv4djtdpZIvQ15N+SR68jsYPoy9e5DheXv+v5z0w0Znlv46cRS5hUrzU9/vBsmc98v8z1aQvZg+8jlfX+dKBMNCb1E/GEti3a23u93XthHjhwgDNnznDhwgU2btzIV77yFf7oj/4o1z2Wknc5ZMUBJvgW4FIsyDz6I04/PT3IViu6tZfhUz/IA5gmgAqPs5pAEkiaZj3y6korj5ASUWvS/71z9P3NSUYQeJbN7JaN3H14O3f27OLMoQ/g81KLEAHswY/eE6rQwc0jm9oK+8JVwN/DB48QlFQrVZIf4Lbig2WF6ANWAThcy58lo/iWZTiY151gVAov78zCGnxadz2d36rPU6rWTldAqdQIIaDaB6vGoDgIsmDeDMWF9279W0pOjQgvk1Rpu8Avfy/waAadnjVhkofHa8XvMW7d42drP0tRCSkyOVnn618/x+7da9i+fVVkflCVH04I/udLu3hzttd/1VWiAKm+xxpmsNQLHjmflrBz3bKgWvW7/+WMI5tnvbrjODz//PN84AMfwHVdnn32WR5++GFeeOEFAJ577jlu3LjBE088wfT0NJZl8Tu/8zscP36c/v5+Y963S1YkYOaVxYaJ0i1MO9yphA5BEqnfwYfQ6rgt/P4lkXnJkCwL0wTYSd8XC76q2PUmj778Bq/0VWkV/OchPEn1/DiV89fZ+Od/zfjjjzO/fqC75SQAJelH3HkYn8vWl3mExyogJU9d+WB5EH+eEeJWah6AC3Vtxge4pCUC6jKBNLHwBwSP4lskOuCquvI69lTxKeIH6QTkMXXgoQNSWK2TKmVSRSgI2NALw+uhUKQdRF53GFJA+b/f+ucMFKf9yE9Jor/DLM9qYRqwCu3T9FN8xRK46Fzmc9Uv88z8U4zKYX74w9v8yq98i1On7tDTY1Gt2oyNDfDoo8M8+eQIu3atQWwY4nMXS3xt3GFaVDv0q8mCr+Nb76m0a8r5SAKzSWpZy29hdhMW79ChQxw6dChy7rnnnmt/X79+PVevXs2d9+2S+4CZILZtL5rb1y1Mu1wGg552NU6o+OGykryAqTcLQXferSZaV2W+lkMqc3WcpkQ4Nq1yiVZPjx+2LRCrpU2G5XDeoOIh/l4d1gikY4FrgSdAWgq4iWSPU11vPz7ArSX6UFVgyhvqbjO+s1BByavPMdZTflsoBfy1mruJhvNTBwYhNRwPoBMXO9D1CD5ohmVJsnSSCJmsyrkW2FGANas6of706EIqaAb7je8bOMLeVUexxCKZoKT5b6XY+TPp1qhkRszz+erX4PVR/uAfzzB9rwGUaTZdZmZcbty4y/deu8cLh6dxfq4P5/Eyc03bZz4KmJ916MGs7vmcNKpNo3QSf2HnnG135jCXK/D69PQ0GzduXLKe/5ZlRQJmN1t8LQYwdQvTqlQQtp2/bktABJSsSO+PdBX6PfKWPsmyBL8t56koSWAbipBQnl6gVGjSdGXgqyFwSwVa5R5a5R76Ll3l9uhDwZUcQR4GWhT/3jys9pCWQHoW0hNIGXx3BTQL4BqehKka9OE7+IRBTnSACy1LHSxNusbwrdQwtKaJis2zbKQHP/btVjoRilSrJAQ7lcJLk0F8elilX1XgVa3xrndBwbd+x/ABuR+QThQY9Tm60CJWsHFN6Q4bK9fyA2YaW5BocWaZox0x4ZYHLDx2hw9e7+HNz/Vz6nPASBm5bwieHMXbuQ56y7jSpq7vva0OEsL54LRg/yZr02wKY0bU6N+PYg5zdnY2l9PPO1lWJGDmEdu2abVaiwokbFlWxP159q238Fw3soQ7Nng0NIiCYmFmiQms8gBdXut1Mc3J1GQLrRYFCc12A5VY9SbFepPi5AyP//v/wPmXfjKqKIGetdfU6X3vJLIqkVh4roXnCaQMQRPcloPbyhmVoM+D93qwSvhxCcNdQtTdQvJaltvwQU6ds1RBN9SVJRV80N1IlG5Q9bXID2x7gf3Ely+oQBaGXOvGWQh8Gvs9+MAeOrGEOsNg9iFYhMtcwPgc/vm2z9BjL6Q5s3bE6NCTkCbzYlSZGszAdBOJwClIdv5zweZ/XmZa9nJXVrndEtQb8zRrLm7LwWs60Cj4TEdLdJbhZIUkTDIUk8DTaIrGE/b0dFr0cjr9dBNz+50o9wEzQZa6FlOVm9/4Bs2JCWwlLF77U5IYS1Z1+kkDtbRr4dKSrIFr0vk2JZsTuU2DATWb44GFh+VJpGUhLRFx8bcbKb10uy+TlDfN0PfYBFRk0KlZwbO08DwL1xM0G0XcvK+w4mG/dwHW+fgoXQvpWQFYCn/OrWn5HV5iuYLPB1wfLPuDztUjoIgxz6Umlkn6IfhGAsX6vGcIlnmMsB78wAahlRpapToV2+3emhY+mD+Mb7FaGB14YtZURplHStcpiEW2v5zAqYNhp66aHH90oIzqCb+VRJNVYppyscl8cYZab4marFCr91BfKNGslZBzJXBF5z3oStWBFZgbUyIlpFuT8V8Csk3HArGiXrqPAAAgAElEQVS9excr7/atvWCFAmY3lOxyiLNImqLbwAWmdtXNC04D1TTv/fTZkqhYEmzPw5ICGdBtEt+ik5ZFaWICZ2EeIXqCPkpG72x7DDx4l1Xb70AxNP4s/09YvqVpCZpegWYblAylUUGu5FI6OIvY6PlgKQNKV1pINwDhpg2tUjyv9lDEcAvrHzSQfbRp4c58avC9KbIBs89D/EQDORKAdpi/TeuKfFuQAayXvmPPUFBY0zKbcC41r7Vj4VvRe/AtS0EUJE20a9pOKEqf/ovrvsyWysV8dGxWAzFcl8H5JEYzrR2kg2bnWmifWkiE8BCWF/fqSxpVmq7FvAUT8iWUOvILBJGwePc3j84vKxIw88hyAGboZetolUinYo1sSiAJux0Z9Zm+L+YFm5x/8jgP6fl0HQIfMDvjdxGcl+BJhOex6swl9v4/X+V7v/DLMQ2202Rk5xV610+BA17QJXltjTYugroote+X2KMEfZsoePT+/UmcBxo+dkgb6Qk81/IBzxG4roVs2Zl9k7WqReEfzkFfYPVK0aaGpQz01YrQdNI7+qpL4X0LsN4L9FhIj2B+NgDOlgWtDBpNSMTOFuKJJl5PEBIndIgKvVSliG5sbRL9h4/hOwuFUY/0EHmqRZlCuyZJxZqnx6rlo2N1ScsTXrNMoKfXFl2Rel1of2oawyhKN/qSrEQTe9rNaNSoyHxT1cJ0XXdZ9rDMu6zknSwrEjDfDgszjBZk2zZOby8IERkkquxLCJqmUoX76eZgQ43tK+0FZ7VF9XtWXNq85bPVAyk7dGyb1ZLYrWb0HNDTM8fY5lP0VBfAUsHSB0wPixY2XrA2Lk9faxVdBh69Q2G0BnZI0bUCcAPPs2i1HDy3GCtPTFe1RfUnJxCrpG9QeuF8agc43aaNm+acAli9Lj3vm0Ks9fBENH9o9XotC6+VsVbVljjvmcfZ08Bz/Lk4PAvPC8A7dIhqFH2qOVMkrJNwQARLbUTc01OPTDPfyZqmVpU+a5pfGv5iJ1jBYqULsDWBpF5knSWNp9UBNeFekYZPHMuSADXpONbo1cwmbtff2qtS6fQMy7UO8/4c5gqWpQKm6mUbbh6tS+LUg6onZ8NPamd5vVtDZi5J72J2PjEdW2jMVNiDBB9Wq8Xqsxewm02aFBDCY+3ANcaGz+IUGsGUoIVLQL9i4yFwcXADODaWU3OItAouqx++QWV0FhyJ55Nn/qfwPz3LxtW9a01TmAWX/oP3KKyvIVWQkwLPE3jSptW0cTMsQqvcov8n7mKvb4It8ML8gWXpOzVZNFrFVD2i4FF9zySF7TWkFeqw8QIQRwo8V9BqFvBMlp/+Avtc7PfVYcxD2n4ZcG3fWnXD7cNEx6LMYlKTAAEoFhsMV25iL8U7Vr8embdMK1LcqkwaVJrAU4MlZAoA5yxQ/uNMiWbo7Y1amMtBydbrdXp6Fhl05B0i9wEzQUw7jnQj4dKS5swMMydOtM8nTlfopl0wf+nYSpoESWvUebxk84jVdQM130OPkx2jflstdv3Zf+Y7/+NzTGzcwo6hY6yp3sSyZNua9GeGaINlgyJe8EujXZ4kdlKAsDzW7rhG36ZJcDodHIi2fheLejtcUPKPtxyX1U/coLRxHqwgfwC4UgqwBa7n0WpmgGXBZfDRW5RGagHdHFC4VuCpKcH1HGoL6dSZVXAZPHiT0gM1pC2U5xUAuBS4LZtGvZTuBSsBS1LaNUvp0QVkBVwsPFcgPdu3nkOvZNdB1gr4e2HmEL0R9OA7I22F/2HmD+m7Ou0n6IaSzXL0iWGhTqUmYVLGQDeBulWPJcEcdHhdHzCYqFjzzcz0biS/alnqSjqJe3vNm0cvVsJVAW9nIPQfh6xIwHw7KNkwv+N53PrmN3Hr9XaM6Ej1DepwUpec18JMLEfG9by0bJ64tGl6w6aaV89meZ5HVl+kZNV9q00BMy+Yq2zh0FKeao7xPAAbH7rE4IY7YBO1LOncxyPdigMfeAd33qZv4xTCke384Z8rfMvXE0FnlPLDh/bfoPeBKaRNZ0AgOmXzhI2XAUjCaTH8xDjljXNIq6MntMalsHCl5XuTZYhdaTL4E7cobqrh2eGzsfCkhRtau9Ki1SjQrIMrcizd0UFyDN9xKHQacmH3X5+g6s4jwoq7lHnMlLwdjNGtw7SZb6GlT573NLcr37pPLFiWtZnLGjWhsXrNp2SXGzBDuQ+YK1SWbYuvUomW60KwpMREnRrreyCFRczFqPfIE5s6CdxUUixv8HX9WCfWssbjSChPT3Dw8EvcefT9gQ4rsPo69Ks/X5lCweo3EAAeD2w+z9DwLSwrfB8dgPPalmUOBwgBQ5tvMfTgjcBK7azZC7/bWNTaYXkSfrSQrN15jcEH7iIs1Qaw2rq8iLVrFstpMrrvIpV1c4EzbcdidhCAFThEZVNmlbVTDD9yHbvaQloSGTx3DwtXWEjLwnVsWm4Bt9lFF2JJrAebyB0uco0dRP6x/HWJLcHIxDU21y/Gn1eSZMwHJ0lnI3ff6jPjkQqOIiGNDpZC+557w7qEghq+mzqQSKI8mSQDA506vhxzmIsNI/pOk/uAmSDLAZi3bt3izp07CMfxKSAlmEFql6BcLOSgZNVsepPpxjI00cXd6AnzmrxsdV1JIgB7ocG6Pz3M/HOPByAZdth2AB7QMkJ4x8rU264lXDatO8+GdZexba8NAFEL00q2LDUGb9Wa26zbfBXHaQUOXaG+Dvi2KGQ8M8mqdbdZv/UaouBFvH7VMjUztDhOk83bz9A7NN0GXdUZSiJ8OjXDahbCY+3ma6wbu4YoSB8cFceqsGwuNnVKinVP6gu3ehuU989S3F7H6xG4noXbsvFcC9e18FwH2RI8MHmOAzdfw5Fu/JknFpooVunXTGlTEyYlsmJpOkAqlLZnHhL60y4Jk6lJGBd+mkbapsZuVGJO3N+/vBbm3Nwcvb29S9LxTpAVCZg/akp2ZmaGGzduUKlUOHDgAN/p76dm2AuzbX0lzWXgW5h5wDLJYSfvC04pApDfSzaL4oU8S1QkdrNBlXlc7KCz9zvpejvqeNZdaPd9ltVi0+AVtqw+F4ClfzFK84q2FSd0HZpUK9M8MHaOYk+jTRfrfw2KmRuFDA7eZfOD57CdFhh0+GVKt3Ydu8HWTadYNXS37ekbgqQ6J7uQYaHaVpOxbacZXHsH0dajA7hNkwJSeYNp7EjP4CyDj9yhvHEeWZB4wn+XLjZS2rSkhefZuJ5Ds+HQLybJt7C0C0mstOm8bR4W1ETF+tcMFmm2iZoMlqa05oIkJIgmFgL6+jr1St3QerEyPT39rg+LBysUMMEHzbSdxhcDmM1mk7NnzzI9Pc26devo7e2lWCxiV6vtNCZHbw/F+FSLJDrRfvJamLqkAV0WSKrp8u58kiZhGkukZxAulI7fZvVfvcXcP9pGnaKRkjQFLdMVW3hs7TvNWN85REDDdqzJEJwsmpjm4OKFLDo1djxwjErPPAgZWG+hY42vs5WjWZVLs2zdfIZKcS4IvhDVEVLQaSKEx9YNZ1g/dANhe3SoXNXitWhqlqX+q4pOjV0bjzDQP4FlhWBJ27p0FbD0n1P65GClPMOGbZd9i7cggwFPSKn7tLcnbDxh07Ac6l4Pbsvm3/z5/0FffRaRPX2sPYgu00eLqyjpULCmG8TPd3SY25ACyjIc+Yp4w0uyIs0FjeZNLY1Jmb9TiRq4AJY+97gSghbACgbMLNEDqKeJlJLx8XEuXrzI1q1b2blzJ+Pj47RaLeauXPHnaiwLcuoLReADZga+xMujfOaxDNNG0aHkXKmX61we8kcsNCmdus38P3owxx0SzgvYXTnO5tJlLMsLUlmBDRPSixaNSHR0U2H8a5Zw2bHhLQYqk0G/J4Jn3AHeFjauqVkFXs8IScmusWvDUfoqk8H2bercpz932aSQCZjb1pxk0+pLCNvfdTnZQk2uARVnjkeGf0BfedKPthQ8E91CrVNsDyri1pOvv2A32LLxDOvXjmPZHlKIGFiG35sUqFNCKM/KcZsI2aWFmdXP69OKyjndQozCjJnrNYFm3Ibz88etUAU4s6zHNCtTf0QxM9aUsTNR4u9Usrxd/30Lc4VL3hHX5OQkJ0+eZHBwkCeffJJCwe9UwmUpd19/ncljx7BbreTBY0of4YRtNgUx0waf3RAtaW22m3WYWfcIrdVUgG1JrFv+yndDXxfrxlQJr+0vvskDzmVs4VOepnk9mTivF31btmixfdVJRvquIgLLUl3i0pm3NIOc6j/50NBxVlfuIITXtgjV+TAXO5PO3dR7kbHVZyjYzaAPjQZy8ME7vXn32rM8MfwafcXpEM3blmX4fEJwS/tFAo/hnmvs2XA0sJg7QBn9E7gUqNETK9uTJ19lePo2IoX1SRWzz40vakxmwxRHOusZ5y2SLM34XKj/KWUXA940K9PTrqVamBg+/e96aLzlkOnp6fsW5rtZsijZLKnX65w+fZparcaePXtiE97hbiflSsW4F2abmg2KEBkkKsUK98TMKzr2Lhbo9KbWrfOQ6Vx4Ppd7Qcuj9/eOMvXhR6iPjeS8c6i/xX5xko2Fq9iYvGF9QKkZaV790y/1A+XLbOk7j2OFA58o9ekm0rpR2dZ3mk19l9p69LnC0LEpTYYKd9jf/wYFuxGxTmkDuKBBwQiY4TuoiDkO9H2fVYUphPDaVqXqBGUr3rlJfbhDk729R9nScxHbCvVYbcBUaeoGJTykXsUBeOjaOdZN38IScmkjs0THHnNC3cqMv/mO6KAZBc6orpilGYJmGp6l7Tmadi12wmxZqjeq1Wap1WoUCoUlz1/CfUr2viSI53lcvnyZ8fFxHnzwQdatW2e0Rh3H8b3PlCg/8fBayUDZ1qPtWJI2H2lqh3kA00Tk6EVZjKWaNBjOrUuCaHYafDoZ5ndNBRrs4Sgj4gYWXrvzV63L0IM1yVbQz40419hdfoseqx78ljB+bQeE8yxFGXLusKP3JCW7TpQ+9e/qYdHIeFsVZnm89zX6rBlAtkFWBV6fyk0G7wJ1Hi+/wbBzGwjrSYeCJQA804BClX4m2V85ymprok1Rq4ESwrLpjkIEv1cV222BlB3v5uWgM3SJUbEdSWJp4vRs3IqMA2mCbZfnxvrNk7z5jPn1FmdKKLFtC2hw6tQparUajUaDCxcu0N/fT39/f5sl60buA+a7XPJQrkKIiAfZ3bt3OX36NGvWrOHgwYOprtjtwAXK5tFJEht5KpJ3x5Ik/d0AZlp8kLyWalqTVcuUqUOCdXeBgT8+ztz/uiGi2cS+AZSZYw9vsZoJBDIAj878ooRgSURPQsmIne8T0+wrH6PfnglsozDKUNRzNEtXkRqPVV+nz5pBBEDXDgbQBuD05ligwd7yUdY4d9sw6+tw2+Drz6Emvy2HJo/aP2SDuA4sPnDDCNfYax2lKuaCckQHJB2Ho0IAmPoT6YBNdWGOX/va5yi2muSIF5EuedjSdjnMzj1qOeMWqImeTR4CRvJLRU+WlZnUEDMlaZjaOXYci23btjA2Nsj8/DynTp2iWq0yMTHBpUuXaLVaVKvVNoD29vZmLjuZnp5meHg4TwHf0bJiATOPhKDnui6nTp3C8zz2799PpVLJzBs6DdkJlGwosbagNYpC6Csg0y3MhOxdA12SqCHtsgbFWXpzr/hqulT/5krC3aKyign28Ba9zLbBsjMfZ7UjA6l0p6kLVG2GEjUecX7IKmsy0OmniETzgUwq1qbFw8WTDDn3NKALaT1BM8lZSCnjdnGGTdZVbNzI/GlI+3nYiXOoAA4tHuYtHhBX2q5P6mAidMhpxFArWoM2c4E9vEVZ1AJgBBks/wkBxEWwQDn1uQBUmWWvPMqgN9Uuk+GWZjEbfPnykWxV6hKv2/mcgdQb+n4KomM1qtajiXoyFVA9l2TGxk7oCaNesp7nUSwWGR4ebgOelJK5uTmmp6e5fv06s7OzAPT29rZBtFqtRoyO2dlZtm3blvIM3h1yHzBTxLZtzp8/z71799i+fTtr1qzJnbdtYQ4MICwr0ujU9hFaU0ZmhSgl242E98gCp7xTI0udwwzPS7Ip2XZ+D5xb89hTNRjo0INR69JjE5fZxQkKtFAp2I4FJ7CCAAjdMH37OMYG6wYEABynHONLP0z6t4iLbHUuYuOGXVbwKYKfaZP1pjZxmR3WaYrtfbI686dhWcIAB0nP/yHOsJ0z7XJ0qFzV8Sjdp3orF9jHUYo0InlCKtdD0MTBUyhqU3mK1NnNMTZzmQcvnsfJvdO3QbJ4dcP1KF9htiT1tPp13UpVuY/E55jLSsyQVEo2DpD6Od9L1h/kmQKvCyHo7e2lt7eXDRs2tNPNzs4yPT3NpUuXmJubw3EcJicnuXjxIrdu3crlJfuNb3yDT33qU7iuyy/90i/xG7/xG9GfJiWf+tSnePnll6lUKnzpS1/iscceA2DLli309fX5O0A5Dj/4wQ+yn9Uyy4oFzCxK9tatW0xMTFAulzl48GDXE+OhhXnm93+fxsxMNmmnWpBLBEy1DXU7h2m6lgd4s/oBvUx5REgonZtg9VffYu7Zvx/TWKTGwxxjA9fbNKffcdvt7/68YBEv8a4m8s1jO6fZLC7iEK5vjIKDb4mZ5y3V5z3EbXaKk1RYwAde3ckntMqSpY9p9nCcKrOoFq4V3E1i0ciwcjdxmZ2cpCfYoNL/TS1U56V6Ch8qkDzAZR7jhxRoRgYl6jIU38M3uVsRSEa5wl6OUWEeEDz+7SNUp+a6X38ZVZx9LoENTVeqAqp5QBI/Nvl1J4jJGEyyNtP+YgqTS2nbHQszb5Qf27YZGBiIbN/VbDY5fvw4f/VXf8V3v/tdvvWtb7F9+3be85738FM/9VO8//3vj+hwXZdPfOITvPLKK4yOjnLgwAGeeuopdu/e3U7z9a9/nTNnznDmzBkOHz7Mxz/+cQ4fPty+/q1vfasrw2W5ZcUCZpLMzc1x8uRJCoUCw8PDrFu3blFeZOF+mI3JSVoLCzhSplblJErWFgRr9bIbuL5FV2jNLdYyVO+ZV0+eAXRXQbikh9XUJ3Ykm7jCFi5QDTZd7ACRP6/oe2k6NCkY9i5OL+UI19jDcRyabWAzWZdmfZ0y9rDAPo4ywLRyNXQK8fU0KWQsIZHs5yhD3EO05yotdM/aNCefASbZz5v0aoAblgNo08GmOiqBUa7wGEdwaIJiIat0blPZYs2EVgWa7OcNHuAKNl577rbYSnB1WozjTx4v2dhOJVGHHmOknthx9Ea6tRnq8ZeVBM9bdo3YycCYmjeJt/X/HEdQLPrvaSlh8QqFAvv372f//v185CMf4Td/8zcZHBzk1Vdf5caNG7H0r776Ktu2bWNsbAyAZ555hpdeeikCmC+99BIf/vCHEUJw8OBBJicnuX79OiMj3XnK/6jkPmAG0mq1OHfuHPfu3WPnzp2sWrWKM2fOLDo8XmjBFrUNVU1VGTB70NFZVpLHCjTdR+2+8liBKmWsSrprQ/5yQf49OgFEzWXjZ77DtZ99hMbaAqu5y3bO0Mc0FgQWjt22+jpLM2zqCNKAxCSrucM+jlJWgFida8yyxFTZySnWcQsLl9ALVgf2rPnPHZxiA+M4bTo36p3rEQZ4N4tNi0d5nSEmIPAaNgFuWkiNEa7xCEeoBlayHlvWP063LPuZ5FGOsIoJoDPAqd6cY82lu0Rqyo/CQzbUm2nwhQniXr2qRzOxz7gl2jnWCxG5oakQnc+ur+sgGf8M6VhYvs2jZ2ZmWLVqFRs3bmR0dNSYZnx8nE2bNrWPR0dHI9ZjUprx8XFGRkYQQvAzP/MzCCH42Mc+xkc/+tEll7tbWbGAGQKalJLr169z4cIFNm3axMGDB9vXlhqAHcCpVkGYR6ztMV/SgFAElGyKO3yWLNbpx1TObppV0mA4tHq7EbvWYMvCOdbyJr3MBOBvt0O2hbFm/eUi/trBZoZzj0l6mWM/bzLIVJA6aon5uZOaTFT3BsbZwiVKCgUa0p8Sgl1R0p/oKiZ4mBPBvKUaDcjX5Q9szIArg/97eYv13NScfKL0sr58RP0lvUzzKD9sW8nx5TB+sAYdtFVLbC23eYw36GUmuNbxDB64O83Q+QmEP6ZYXjGsBOnEPzdblHEF/qfJwOuAY3zCNI3CjUkaxqV9jx2bmQ79s1Tq1OHl2jx6ZmYmQteaxLTuXZ8aS0vzd3/3d2zYsIFbt27x0z/90+zcuTNG+/6oZcUCJvgv+cSJE1SrVQ4cOECxGLUclg0wLSu1zkMCaOJ7yeaxDpN05wXLrMEs5Ac6XZ/+PQ0mYuWXULo9x74XX6H4bzYEwdiddhQaN/B8DecUPYoxYgzMM0qqvVCixj6OsoZ7iPb6zbgV5cb0xJ9ahTke4Ugwb6l2yC5qZJ80cWiyl6P0MQtIpRyqxZvefNdzne2cphDsFK1buDKgUZOkhwUO8H1Wa1ZhNLqRQCbSsDDMTQ7wA8osGOdue9w6llSeRregaXLuSXL4aV/vAFkaUCa1OXWtpamuG28c7oOp07JRxcmNRj/W514iX5IzC0EkLJ7ruotad6nLwsIC5XK6V/To6ChXrnS83q9evdp2KsqTJvwcHh7m6aef5tVXX33bAXPpIR7eoTI/P8/x48fZuXMnDz/8cAwsoRN8YLEihMAqlRBWZ9+9tIGkSdTg63koVYhSqhbpL7kbqzWPx21yJxPV0w2QC0/iHJlkmFsMcZdV3GOAafqYo8o8BZpYqb8k+8nt5Rgj3KBEDYcmDk2KNCnSoBSsJtTFDL4eezjGIFPB3iqNYCViuELSjS6fSCjPg5xjPTdxaAZkp1+GIk1KNHEyZj5LLLCPY5SZD2L+hL+nSYkGRRpBOZLfxD6Oso5b7d9RokEPjeDYL0OY2zQgHOIO7+dvGGCqfd8SdXqoU6ZGiTpbXrmM1dTaWJ55SD19F5JuKUbB0jSwSTbyoojd1mvCrqS/tBthSBdLk22OqpTscm0eLaXM9PU4cOAAZ86c4cKFCzQaDb7yla/w1FNPRdI89dRTfPnLX0ZKyfe+9z0GBgYYGRlhbm6OmRmfoZibm+Mv/uIv2LNnz5LL3a2sWAuzWq3ynve8J9VbdqkWpuM4iFIJaVmE+2EaGRdpGCQGkjdwQUyncpy+ui/f+dAyzDOfmnWt20DuXhMaR1o4x2ax91Ta3qH+HKCTkCsuZmNEspejbOEidgBk6hxj6AWatZA/lE1cYZSrFGgGpenQsKEu884rHVnNXXZxkh5qEYtQDZqQtsLfpsVuTjDEvSDaUfhkQgsXBFZkCBB9cpIxzjPGhcDxKWpZhsHm9ehGqo7V3OU9vNq2sj1FRzgXbGOx+tQkdsNbWsCCpAZieOHtMnaxO0caG6CCpdSOI/OeaWxpuuLOcVIDj53UTdHOeSGgt3d55zDzhhh1HIfnn3+eD3zgA7iuy7PPPsvDDz/MCy+8AMBzzz3HoUOHePnll9m2bRuVSoUXX3wRgJs3b/L000+3y/zzP//zfPCDH1xSuRcjKxYwIc6f67JUwLSk5OhnPoNXrxNyrontJgBNU4nsHAiTVGU98s09msqlR6FcTLMylSvPVmF6s3fvQP0tl+IePaU6E2XWlkSwCTwe4BLbOBcAnDpnKZC08GPOlpQ8yb+sn8lguUQHJFRPUh8wVWSI6yhSZw/H23O1aqcb0phpNCrARq4yxgUKtDQPXz/8QtzLN/rkVjHBo/xQmX+NOhllbWFWZp4neI1VTLZ/ZQiU4VZoPoVe8OcuTY/DMP9olCT/m0znnqgCPaSdXpz4PGd87lIqn6balmgx6p96IbyEa4miAqWq3Bedkl3qHGYImHmipx06dIhDhw5Fzj333HPt70IIPv/5z8fyjY2NceTIkSWVczlkxQLmj3oTaYBCTw/S83BdFx0AjFU5YeTpdOH0Y0rjkH8eNCk8Xqgn733TLNfFAK87D3N/1qT4szLF8bW7ofsoV9nDscADNb5hskwBBr32FGmwl7cYYCoCdOruH1lAB7CFi6zjZkC5Rh19bCUwQJJUmWUXp+mhFsBjx8M3BN20NZsFGuzjTSrMBTmsSBkIwC7puRRp8Bg/YB23EdD+/WHsXZtwfWyJoR9M0HOnlvlMlkskaC8uOvzRrcM4KEZHrrp1qX6PDoaDczKhz0myHNPmOOKFzplY0tvbGQAuByW7sLCQK/rZu0FWLGACmTuWhDuOLFZs28Yu+ZUzCVB0KzNyImhfTja2G9tceC5vc0gDuW70mPKrP6ubtaGqktphF1mLAqZJTx6XjlEus4Mz9FALUkfXR4YBD6L3SB4WbOEiI9yIAV2HPk53sAEYZJJtnA2oWIiDlYeXsQxlFycZ4g4W0qhDZKzZ3MFpRrmGpcSnjf75O5IkvcFdnGAj19vuUZ11mmEoQZsWRUBQvl6jNNVEpMdlT5a0dDoda7A6TTVEtzxBb1u6w1aSRWkCVBFtnGkmbeKelwkFb19IiwjtU7J9fcs7h7lS9sKEFQ6YWbIcc5iWMvJKbCfBCVMbEHQf7UdvLlkvWQ94oOvIqycpn/59UTufeODdltS+Wsf6SDlXn5qUZgNX2c0JKizELMtO9Jr8TWM193iIs5RZUMBJD3+XPUm3m+MMMtl2YIp6tcbD8Omyjpts5jI91JV61hkARD1a47KGO2xPGESEc5CtlN8xyhW2cZaSkj8K2P6zCFuUcIlWkK5HUcSB0fSpSJyDMDv1xDEpyfXWrD8CuMljt06mpHiypJwzF9RwIQqa/f0dC3M55jBXyk4lcB8wU2WpgGnbNlZPj+/0Y/C2NVVpYzms7DThdZP/ZTcvOc3KzNKjMknhsak9L8rCBGQTmt9tUvpI/M7mPjIO3+u4zh6O0xOAZWeOsBPFp04hN2AWaLCfIwEV69twKg2pb+Sc9Ls3Ms4IN9reuBKfglXnL9MBU7KPo1SYD6jYUDo60qIKFamzlwbcblUAACAASURBVKNK8Hr1qfo6WliG/H6aKrM8zmv0MqeUufNMQ9AWgXVr1T3Wfu8ewpPJbtOLqSRLluS5St3SNEloSaqDBKlelNr3NIA0jT4TrdKk1h/P1N/fGfR4nrcsgHnfwlwBkkXJWpa1pE2mVQszrX3EREvYLSWrHy/2Jetl7iaebNrCCSvHhKrxcgta36hTeLMB+/xRcj5L0++u13ODRzmCQyuwquMB1V2cdqeeTG11ZDtnWMvttlWo0rAdCtL0Bjp6bJrs5wgV/K2yOl6x0c2vk0WyjbOs5p4S4L2z4CME3DTZwgU2cCPIL+h4tfrl6Hi4mu+/l2MMMNX2wVWdp8INthsU2/XJangMHp3CcmW3wZiSJY+3LAQesipHGx9uSeU4ydJU30vi+1EaUbsrMW3hRco5TU8+MaOrEIK+vg5gSilz+XOkyfT09H0L874sXWzb9rf3EtFRq8q+tGccwgZlaBCFAKm6he5Q3XIsK4EAuBc/fmjrzWNhGm8jQU5JGG/BvnzEGPhuKlu4yg7OtOPDqlZlCJ6NnJZleN813OYhzgaReCTRjZxD3VYkT/y7yw7Ot8EmSuOGll16mcrMs4djlAMqVnVaku1weMlPq58ptnOGIjWEAnRJjkL6u9nEFbZyqT0QiVrtBI4+xUhe4WZUpMVy7qa5ywxRn7n+nExgGR+cmsA3p2QBocmijHQcpgxJ3yWWFV2HuRxyn5K9L8siU2++yZ033jDSsbp4iQdQTPBIN4mpueSde0xji/yOb5FAp523uwTdSDlmQP7LKfjQYPuKGrFH77aKNNjBSR7gKnawJlGN3BOujXSNQdqTxabZ3nGjYwuG4OR3onlizg4ywUOcjVCxYacdesWmUbEWLts5TS/zARms04E+4CbVQIkf87afmXbMW/U3QNRRSH91Bers5a1g55HQKzcasF4vvwT6jsxgNYJS5QS2lIfQhXXZKYP/mQR08WUjqkjjuQ7FHz8fpE+iW3WRhr+kNImZwnMddFV3KlkuyRMW790iKxow81IRi6UtCqUSrelpbM8zVuN8hcw/hxmmCZtIKOGykrQ84Wcalbr0eCC+ZFmYacvOpAQx3kJcaCK2lkh+opJ+JnmYE/QzjYD2GsCOQ47fsTcoBVQsCbr08rs8zImAiu1s/4VizXmITAwo0GAHp6kyF4BuFKh8AEqnUldzly1cbAdKiOYP10ya35zED133AJeC/S39PHo5krZGk1Kym5OslvewhRIgQURD55kAf/jb9yhOt6KVaqnAqeoxHQuQIm5FmhJGYUYfkunzmvGaEwFP2flM/YFZI1bT90RFpnYRtTCXg44Fn5IdHBxcsp53gqxowMwjYXi8xSzuLfb1gRIWT5dIVU5AqtBLNg8bmgR2eZ11sq7lDWmXlD9sxovyklV1TULvZ+9Q/51eY+qirDMmzrOZi8Eay6inqBuAZguHljaBZmb5ok9oA9fYxnmF3tUdXPytxeK/Ivp9A9fYxDgFGkoZ1SAB6U+rQIOdnA7izaLk78yoqoCr98E2LXZyimrbUSga5KCzdZhZBr0JxrzzPtgKOlMPIhg4CIEU5tpn6+Hw3g4R6jNIAr+EjJHrccvTZHHmEh3XshpjYnrTkDyeQQjo6eluL8wsmZmZYfPmzUvW806QFQ2Y3QQvWAxg9vT1+R6ydKqtHn9DQtsZIFb3wzKEbTqHiRm3sxbnlWoCurxB05OIoVC6LU+sj3DB/uo8o++9yrmnx9oFK7sLbG1dYj3X6LFqYAeWggi2/xJ229vUDXY3SZKk6bEqs+zkJOVgqyvdIlPXb6bprTDPNs4FAQa8mJ48lO4IN1jPDQq0FEuoo6Oz5tEsm7jKSDtIQvhbOveXkLB1mMCSLo/U36JfzoOQeKH1JvA3GxACT1jBu4k+TWvepXdqHmGqz4s1eJZonUYtw/i1PAxpVFc08g9q0IJkUiR+3oSBpryxBCaFEtsWVKv+IHE5AfP+HOZ9AZa2tKQYbu0VxJFVxdgmElpiuCdmN6KqyaJk9fSmYwAhu9OTNNa1utRjktZdWPvtaQ4u/JDJfVWcVZIK89gOyAJgg2uDZ0PLFkjHwrUsXMuhLoqZaxpDUctp4bKHY6zhLurSC3XNpT/flz2yGeMca7hLZ9utcClISAeml69EjYeCNZPhWCqyhKEdXccsPSzwEGeD5TXhL5URPWkOUOuadxhp3KEggvIL9c/FtQRe0RyRYOjNSXq/N5/O/y+3GB5FlkWZJ13c6sxwADI1ChPO6edMu5NECpbU2ogcW5ZoW5jLtRfmfS/Z+9KWpQBmT3+/T0spcWRTu9IkwLSiSUzdYNpgtZtIP/ogVrWIsyjZPKNvMnTkFVmHub+Env2w+tgcVIE+/09UgRJYDlCAYlHiFVzqZWnoxPVfkGxdbuMco8Fmzn5qdc2l1d6XM0tWcY/NXA6s1DC/Toem69nEFYa4095YWrUuCcqVJlu4xFpuYwc7juhl8GJrLjtAIKTHnvlTFN2mv6cAdGKZh6DpJNeCtbcnfOsyB2OyaImyrbGX2vGKVc+pnzoR37kWvx519onhmQwekqd5HJlubLpmKqRJT6op6p+zLNGOJXvfwuxeVjRg5qFkFxsez/M8rt25gwfYAWgmAVG8gUV1dWth6vrSKNCkTsEkb1fw9Vz6JLjj0DoLzl7ABeqB4hZQDv4kfui1SDg9869Om8lazV22c0qLgqOuubSoGxcUxvXtCLxSO0HVonRsVgi9HhbYwRkqQQi96NwnmK3LznFVzrKZy5RoAF4w9+i1dyGRkEoHb5y/Tv/CjG9Xi+D5BuDU9mtxzL++Oj1P3+RcOljmpVd1MEwCSEO6KA7FbxgfOulpog5CnTZnupmC3kbrMOE4KW2qJA8BQAZesh1K9u3aPPrdIisaMPPIYizMe/fucfLkSZzxcUQKFRsey+CLkWEhf/D1JIYnj2WYFnQ9lG6ALq2s3QBvWpnkAjTfBHu3oVzaQzBZNOm/pZPYocEjHKEffz8+33rTt+3K15RGuM4I15Xtv3QqNBswt3CJAabay2T0BfS+V2yyjs3eJVYx4cd7FYAUbe9RCchYcNeo7Ji8gONqS0KCP3VwYqrra69NUPhPLWhqRVzsHGQSHZAgyTiVrUhNq1qaJr3qOWNbSBotJwGl/t2oXL8Qr/S2/aOxMO8D5n0BugPMer3OqVOnaDabPPLII5w5coTGxARWsKwkSVSrSRI0QRUwQ6RSr6fp0c4txrvVpGcplqEq3fSNqc/Ng+ZrUPrHAQ2blCl1pGHiwXwR+J6kuzjNGm5jtVN3FvSHAOXl8EX2dZ0I1iyqwNsBvSw9JRaCIANNpWtUHUySl5EA9HqzbGldpkCjHfBGihAo/bfSspIpjU3T1yg3/J1QjBaeZs13fr2gPF1j4M4M4i4+I5D1yJajsum6rDSnHhNNq6+r1C1K03flfcjooDh1BGiKAJQElGF6abqY1AAkxaKFHewXuFxzmHNzc1Sr1eyE7wJZ0YCZ10u22WymppFScvnyZa5evcq2bdsYHh5GCIFTLreXlUD62C9yrNX3pQRfh6VZhrql2m1+/XwIvHl0ZA4yJDAH3stg/aIhU04qK+3ZbOESY1zEptXuPKNzhdlB0UPZygWGuIsT6PKLGF3ukgaYAo8HuRCsKw33tQz3tvT1ZZVlU+Mqq5rT2MK3Lj0BnhAdhx1DBxrWTcdtMTp5A7vlJtOgsYcZnPAkA9dmKNRcEqModCtJjGmKz41JSbyaJNGvcTEPUnUPWaI+f3oFTwoBm3TD5DGeplz982/iOL5FWK1Wl42SlVIuC/C+E2RFA2YecRyHhYWFxOuTk5OcOHGCoaEhDh48GKk4hWoVYVn+HJE2h4nyvX0+oRHYgrZzRU4MiOZfRJ5QdKDL6oNMI2+078qUV2a/YNIVSecCR4HrwNaMwimSRLapMiKvsYsTlEXonKPubBI6yeQLM1Zmnoc4p2zKHJbCnz/0gS9dV1nOs12cUaICRanYLKejklfnwdplSqHTkvANLhms75AC6pFlINGnNDx5l975uc5ZHaQsYzZAUJhvser6LOIKUNPS5cOmZMmZv/2WRfzGnRpgOhdan0nKow8iuY0q+XWwTBvo6SCZCpb6X/R6uWxz5coVZmdnabVaVCoVCoUC/f39lEqlrgMZLCXW9jtR7gNmhiRRso1Gg9OnT7OwsMDevXvp7dUX0YNTqSAty1i/9bovTfU84F9Dp59uqmY4psxLyZpAXG/LeeZC1fxqOdQ03QRxTzvXvjAF/A1RwDQqSQ9hrkqfnGaPe4x+Me1ntZQ1jiLcEDq++4dZv2QnJ1nFJCKIBhRuqKw66qRGIZeSHfI0ZTnvRxcSYSdO4LST3FHLIP9o7Qa97rxPLXeytgvuRUz/6C8pNJusmbyH03Tb9TLylzGaWn1uFmfWhRPABOmVoJs+exEA69fPzkBDVxZ9jlEATXINC6/pjKlPy4bmO8nbeKXRTkkjRpl4oCkISiKgv7/C7t27ATh79iyWZTE3N8f169ep1+v09PTQ39/PwMAAfX19uS3Q5YgY9E6QFQ2Y3QQuCEVKyfj4OJcuXWJsbIz169cn6nHKZf+atqwkaaAYoWSVi47SGWVZZKaYzHnXYSblDyVr7jHx92jHi21aSc9PusAbwE8CDy1SuVKuHq/Go7UjDDKJEJ6/KN8S7cGPFALPsmmKAubeTEb0rZb3eEBcpdi2LsNOV6Vi04nqPjnDA+5l31lIQCeqjt8Ze0IgRDIK2dJlx9QFf91nwFgItV6pFqLhqfTNzjIwO4vwIqejgJnQmxSmW1RuLPhUbJPo49Irw1L6XRMlq1VaabAuVZGxzKa323lYuQaxmQ0+p6TSLkm8VfR6uWzjum57p6bBwUFWr17tp5CShYUFpqenuX37NufPn8fzPHp7e9sgWq1WI/1dCLIrRbqcHXv3SRZoqoA5PT3Nq6++yszMDE8++SQjIyOp+QuVCgQUbdq6Y6kemPR0aWHqTSdv7NYsXUuZw1QlC3iT9KVStLPAX0IsgrqSMH5PQ2k9j/2zxxmuTdBTdyk2JKWGpFT3KNZaFBoudo5g+qF+W7Z42HuLijePI11s6WLLFrYMA/O12sEL0mR3/RS9jRpOy8VpuVitFrbrl8XyXISXrmPr9GWKzQaiiQ9aTfxn1fC/i5TswvNYd+suTsP1l+y4wZ+nfU+Q8uUazrznbxid99HlqSDLlaZt7etDHZ2e1Yl8c01eEkmpV3IT2Jo6k0gm07FECKhWi0gpaTabzMz4Xt+tVgvXdZFSUqlUWL9+Pdu3b+fxxx/n8ccfZ+PGjXiex6VLl/j+97/P66+/ztmzZ3n55Zc5ceJErr0wv/GNb7Bjxw62bdvGpz/96fjPlpJf/dVfZdu2bezbt4/XX389d963U1a0hZlHQqefEydOMD09za5du3Iv0nWqVUSCE4X+XULiHGY3wddN9wlLkDVnmHYuL7WbVZawHHn05Pm9bSrbBY4Ffz+RnkefegvF9lz2TpxnpH4Ty/I7GKyO9SWD+b66LWL6dHIvlA3uddZ5t3FECwRtj9RwXtsTAhLirYYy1LrHuuYdCtJFCNle1ieFn9+1wC0kj31LrTrrZu/4QJ9khdnJzF/v1AL9E/M+qJoo2CC/ScS8pOdGA6spYQ74O8zgqleK5Wb4Ivrjb8wENdGMqrUZLWhnLtn0cDWFaaO+8NgUPzO9kBmJOwn7+0u4rsuxY8dYu3YtAwMDfj0MBlxuMBgUQmAFDov9/f2RPq/RaDA9Pc13v/td/vZv/5YLFy7wcz/3czz55JM8+eSTHDx4kGKxs47XdV0+8YlP8MorrzA6OsqBAwd46qmn2tQwwNe//nXOnDnDmTNnOHz4MB//+Mc5fPhwrrxvp6x4wEzbRFpKyZ07d5icnGRkZISdO3d2xdW/9Qd/gFur4ZDeXvTveoxNW3Sa3mJGr8u1HET16eimHKa0eTaR7lrqwNeB7cCoVoD2A5bGh+G4LbbcvcrI7G3sMAZgAAwhcAob3JJJaVwEvpPN7topKtTBCj1SQwD2462qXqlJ7+jBhUtUWjWssGIoNKolJDJjh/F1M3eo1hc69UqnKg3Z1V82fOUeVtMLKqdBhzr1qk4dCHDuuRRvB5ZpDR80l/u9q5LOuKZkiJ+TxjRJwElwPrRWg09pAtRIhpwjw4RPYwJjxGpAYlkt3nzzTXbt2hVbO+l5Xhs8Pc9rg2f4aVkWQgiKxSJr1qzht37rt3jjjTf44he/yL/6V/+Kw4cP88d//Mfs3r2bNWvWtPW++uqrbNu2jbGxMQCeeeYZXnrppQjovfTSS3z4wx9GCMHBgweZnJzk+vXrXLx4MTPv2ykrHjCTZHZ2lhMnTlAul6lUKoyOjmZn0mTu1i3cWq27KYyExmPnJM91JgcWtzuI6dxyBC7Qrd5lFQncAL4GfDKtINGTdqvFphs3WTNzD0d4UQtK/Z69vaVyC8nW2mUGm7PYgVXYDk4OeJZLy7HalH2SrK5PsmbhHrb0ohF1oD0XmdSKJVBsNlk7PYGjOq7pgJnSC5Qna5RnGlitKFi3P5N8lYL1jr1v1LEa0i+MPn+5nKJXzKSKqgJ6rDj+RdUvx9SeVEkz+BJ/qilhmvWYaVkmnYye6+93eOihIk888YTRmSe0KENPf9XqTLJCp6am6OvrY/v27Wzfvp1f/MVfjOkdHx9n06ZN7ePR0VEOHz6cmWZ8fDxX3rdT7gOmJq7rcu7cOe7evcuuXbsYHBzkO9/5zqJ0lfr7284ZSVMTqNdSWmieaD/6fULmyyI/aIY69D0p5SJ0pMlSJs9TdXv4y0z+C/BUegaBxKm32HjpBoOzc1hSdh5WCE4WProHYKlGDEpyGwHobc4xNn8ZR3ptKzUEOQRYnh8cPklC3ZtmrtHTqPvHCtCJUGdGC+6bnaE/dNYRhr+MF9F7Zx6n1ow+Qx1wwfgwnOse1oyk7Up8legcZhI3n9dKzEqnlS90VA2l01byDgNNwdY7VmU0jWKJhqMkfccSM2uaj4KNNbJkhbYt2LKlh09/+gAf+tCe3CxZCKCWspZctUIXFhb43d/9XUqlUpIKvyQGBk8vQ1KaPHnfTlnxgBk+fCklt27d4uzZs4yOjnLw4MElv5iCYTLcVOclxLf4Cg+CImSwbomS16JTQTIU/bjb4Ou6TvV8t0tL0o6l+kXid9DfwKdl/348owCQHqXJGsNnp+mZb/jWWwiOIZCEf4n9gRmJbc9lbOYyPfUA6ALwFQoIq0s4kp5pf22G4fm7WJ5npkJTrMuweBvu3PGddUx5IdVqlp7HqvFZrBDwTFN0SYWXEvuGh7UgOybbEXxa9kflVJkHONsn4iZpko2mAmBcYcaoQQXLLJPVBJCmtInWZjxDsQjve1+RX//1LWzZMsD09DR9fX0REOxGwnznzp3jox/9KL/wC7/AJz/5ydQ8o6OjXLlypX189epVNmzYkCtNo9HIzPt2yooHTID5+XlOnDhBoVDgiSeeMI6YFrM7eamvr21hgnmNcuQeCecF+aL9mGkm/77d7GWZlCbLIEnLqzdlJdpf7jy5KS/wvT//EJ8ufF/0kqi5DJydo/faAnZoeakWZfjnkAKWyTI8f5f1s3ewwodmokAjeuNvQEiPDTO3fM9WMINURusdnJyhOjvve6dGjSBflEnpyLMM6vnguTl/7WTS70irDC6U3vIQDSW9bl2aZDmNh4QpQ5OYQTF6HE3TeRjx+qlZm2lWov6pUzymghqBNKpYCBgaEnzyk2v4p/90P81mnampKa5evcrs7CyWZbWXigwMDNDT05Orf5NS8id/8id89rOf5Qtf+AIHDhzIzHPgwAHOnDnDhQsX2LhxI1/5ylf4oz/6o0iap556iueff55nnnmGw4cPMzAwwMjICGvXrs3M+3bKigfMa9euce7cOXbu3Nlej6SLZVl4ntd1+CenUkGkBC6I1fsU5FqMp6yaNsMQyaVH0h0tm6Y/j540tipXhjo+aE4APwdiStJzvknf9QZWI7D4bDqWZfip/nUppWad0YkbFMJwiqqnVB6gCaRcW2Dd1B0stePsUsfw9QnsupdMoab0j1bDo3KjDi3ZeZ7qvQW+dWoAcgnYp0DUiVZw05KSxQKkCcBNeg3Rh3RwTBqIqQCoD9zSQNJPr8ag1UckhpslnTOBaSxd9IJtC3btkvy7f7eZf/gPHwCgp6dIX19f2xej1WoxNTXF9PQ0169fp1arUS6X2wDa398fm+ecn5/nX/yLf8HMzAz/9b/+VwYHBxN+VFQcx+H555/nAx/4AK7r8uyzz/Lwww/zwgsvAPDcc89x6NAhXn75ZbZt20alUuHFF19MzfvjkhUPmGvWrGHt2rWpFEW4FrNbwCwEgAnxRpbEvEhza+yaktVVhExjGuDkAePlWovZLRYl6ZyS0HBhrUzoe13gFbC+Cz2bgSHpW53BJtNtoHSC7wV862+RXkkjd2/7c4amDlvk0ys8j3X37lCqNaN5k0BCAapQKlM1KtMLvrOObiGaJrW1h1e816I42US0tDRhJUqrjw2wbuHnDQBfXgNuEPMAj+hNKU9uyRgIhOI3s3jCuDer0NqriLZXpbBJTAjgR/uJ3jyaYVk205ZUq/DBD9b57d/eyfBwckB0x3EYGhpiaGgoKJ+kVqsxOTnJ7du3OXv2LFJKzp49y+3bt9m6dSu//du/zbPPPsvHP/7xrindQ4cOcejQoci55557rv1dCMHnP//53Hl/XLLiAbNUKmXuRhICZtbkti6FSsVfPqCcyw1YWsJCzg48bHv6Pbt90THGJ5ClxqVVATwtXd5rVz14awF6GrCuBsMNWCdhuKj8ZonvoXkV39qs0tkvs4gPkOGn+qBym7a+9M7OsfbuPexW0PupVG8SUBmkVG8wfG8C0ZJxgAxBN5UykPTdmcWpGTxjw+9pYWulpHy1gV3TrEu9DElSA/sikWcmF0DMkOs5diV5gFUbXMjYxYQRiEGSYgaEwBkb70plbtTUqLJN1zjAGsLrCSEZHm7x679e57nndncNaEIIyuUy5XKZkZER/zaeR6FQ4K//+q/54he/SKFQ4M/+7M+4fv06Tz/9NI8//nhX93g3yIoHzMWEx8sj0vM4/qd/itfyN1tKaiuJJ7SLRWHWkapPkW5etKd9V9tpHks1S7qhdpPyq3qmJcy6MLUA5xvgTIO4Dhv64dBeLaOH7xQUzq2FyyLSzINYAQxdrusxeuUGxYVmNLa3+pcvTjsbr93CabSilqGuL0VES7L6muaso+c1lSW4Zt/zKF9sdOYfk0A76f7H8CMKqWl0Wtdw38TjxBspnzmfTfQ1C+2aMH5PvznEdieJ3Eef+8RMr5JwbAJOLW2hAPv2TfO5z0keeWQso9z5ZWFhgf/4H/8jrVaL119/nYGBAW7evMnhw4ep1WrZCt6FsuIBM48sBjCFZTF54QJuo9F+yPoAUf0uZawfVpR1v8VXeL9QZVZfrQ9ak7C7G+/WpH4Buqt4af2KBbgSGtKfssQDIaBgQX0SJuZhle6VmTlyyRa9Ox26PkV1YsHf3VK3LFVgzpDK9AJ99+Y71qUeUUeQ8vD8hKsvzeDUXP+l6kAH0dGKAReKl1qIuvTpbJV+Dcujb/ysygIwiZ9XrSymNZhpmLRYSjZJT/vTRMOab5wMmtHlJamDWJ2/zTMw0y3OFBq7t9fln/yTcf7tv+1ncHARHmoJcvz4cT72sY/x0Y9+lF/+5V9uW6zr1q3jqaeeysj97pX7gJlDFgOY4AdfV71kdTGBkomWFfiAmdeyM+FBnhedR3cewDS1c113nj0x8wzAwy4t9CcJ10m6EhoenLoJB82+XOabph0nSGmmwdpLk9gNjUJVadicQQ+Grk5SqLX832Gat0x6kaF1uODSe20Bmkrl0QE35eGLusS54WGFAKeXI8vlegrEDdCxpv4XUGqSHAVwOeYxdXDMkTgr2Ho0ck9ch5rWi5wPdynxFUpTA08DxKTGE6QXFoxunOdf/8vb/PzPr122tYlSSv7gD/6AL3zhC/ze7/0e+/fvXxa97xZZ8YCZl5J1cwfc7khB2YU8CUhUkXpCVZdQ0mSIPkCFxVl0pjbbLfAm6VquSD/622vfL+ikXr8MD2+Bvh/R2j+rJRk8N4Uz3+oUKC/IaVKerFOdrCGa0gx0OXT13GhQmGnFnXVCSzXjwdu3JIVb0rcIY9YZ2WB0An/0opczbRscVRbb72eBbaTsQmtm6VanmiYPVZv5M7Osy6TRYoDITlHy5P7bPP9pyfaHhrPulltmZ2f5Z//sn2FZFt/+9reNWxaudFnxu5XkkcVamMVqtb2dUHxuI4GhSUBUO4xpmiF6hJ5Q0rb4ystIeixthKWWLS9gZlnnWetC55tw7IrhQp6bJNy0/RylpPfSAr03av6cnbqTR0s5zql/4Oosznyrs6uHqi+HHtGQlK/XsUI6Vf1rkSs0XfG456fzlL+sMoQPpE6HjtXFtC2Y+vk2iNTulcyAmCdEF7M7iUrdplbmJJBUOwgBlXKTj/zjE/y7/+UcpeIEk5OT7ZB1S5Fjx47xoQ99iJ/8yZ/ky1/+8n2wTJAVb2HmEcdxmJ+f7zpfIdg7Lql/NlmCUk8UlqGLjsXULhe7HERvx4v1bu22PEaK2nCcFd/W9eDsOOx5EPryTvGkUWTKtcKtFv2na1g1jf5UQ+ulUbHKpHVxokn1Vs2nQk3WJWQ+NGfapedmsBQkj3WoH0+CNUHUOlXTZlmoV4G7xH6zewtiO22b9KcdJ6VPy2eYv5XGUac6qNVB0nTOsM4ysdBh8ALDC0ijYrRrwoYtI7P89ifv8jPv38z8/DxTU1Ncu3aNU6dOIYRor6HsJhCB53n8/u//Pi+++CJf+tKX2Lt3b2aelSwrHjDzVCrbthdnYQaxZEPJQ9UkpXFEp7mpc3d5JLTo0vKYaAmpuwAAIABJREFU2qvJhb4bKjUJ9EI9eX9DojWeQ4cEpmbh3GV4JO9cZg4Rs5K+o3WcGQ0soQOWeVuXJ+m9VMOZ0xx1VMehHA++eqSBWNDKo/blGTqsW2DNEn3xaqXL4qOuYLSUmifBmya3l3BuScOrtIqhbSIdX6URjxmbTN2qc5zRnUkkwgyUHYXx52U4LhYk/2D7Df7vX3PYNOJTsL29vfT29rJx40agE4hgamqqHYigUqlEAhHo68hnZmb41V/9VXp6evj2t79NtZq8bvO++LLiARNI3eILFk/JloI95NQ2YApqjvIZgqaInMwffD3puBuLLnF/WoK51JRCJDDKMX3LAbxhH541EHAlvHUGdu2CUqWLG+sFaB9Lym+2KNzw4hHuVQvT1LoMD8iZcqmM16Jh5HQAzpx79HAmXZ8aDvOrerIiF7lg/RDzIl5Vn0nCa9cwL8DvhnpYTtGtS6KfJtGBM/16/FpqAVQFpg7A8NwHq01+5b1X+Y1fXJsaOMUUiGBhYYGpqSlu3rzJmTNnkFJy/PhxFhYWGB0d5dOf/jSf+tSn+MhHPvJjDWj+TpL7gJlDFj2HqW00nTRnYmwzWgNylOCraRamriv8W8rmz6ruvHryUKnd6tCv5TF6wrS1Ohw9Co+/V52DzJHZUCDnLXDOembHGoLPck6dLUn5XBN7TilMEiWbIoUrHtasVNyFtfxZOq7jB0bPQoMk9vEo/pISwwuRWf5yP4b5TLNzjhp4IOp7YOJ6pZJHvx4B5zxIrYoE25JsWzvNZ//JNO97dH3OjEoJhKBSqVCpVNqBCFzXxXVdvvCFL/D8889TKpX46le/yvj4OE8//fSPNeTcO0XuA2YOWTRgGiL9qGICyiRDt5t1mKZRcJ69LPOMnLuxVJOsaC+jPN1Y0Xn36PQknD8Lm7fB2q05MiQV4ho4R/DjpOogGQKTvvA2RewJSfl8s6NPp2KtHHrqksJF6VuooXRDx3rAzeA3dbPnZ6i7BczQebGq6jlonge7Gx59MZIBulK7ZgI5Y572tXy8b2xes90ItGP1Bhrt1FP0+ODoNf6vn+9heHVnI+alyuzsLC+88P+z9+bxUdXX///z3pnJZJJMEgIkZCFACJDIKvvmUinuioKySaEggiKiglYslZ/aqlhx+4qfWlq32qqttkJLEUVqSIIKRgGFsBpDFkICZF9nuff3x+Te3JnMmkxAcV48hsnc5f1+3ztz7+ue8z7ndV4hOjqa/Px8TCYTxcXFfPHFF9TW1gatnwsZIcKk61yyhshIVXzdkydGC1W8wM1KvUBbTUUv8GTFdsTCdG0rUEvV3YN1R1yy3hDIMVmtcOALmJwEBndpJt58vwAVIO4BlPgvrftV+dv1ivL0hQDYZIyHbIiNOLsNtFahHwSmOwG6KhzWpSeXrrcT1QjCIfzXM3XljhbgOI5jcDl+uQVsRaBTKpT5+sKCRaZe2vH8YOiODF3nMN1bpq6dtnmKhLZ3b5am7Li+e5paWDX2JHdOiUffEbUSD9i7dy/Lly9n5cqVzJs3T3XBpqamkpqaGrR+LnSE0kr8QKeECzSajp7csG7hEomgCBd43cdN+8rfgZT38kbqHSW6jlqGvo7V33aUtmrPwnd5HejoDLAHqMRBEBYc6RcWHK5MZVkAbl7xLISdkNvSR1zTSPxJSbGD/kBr33bap4L4Q4In8B3F6grtST+BZ7IONP/STxe0xzZc99csc89TrqIEbaTo7HJVtm/bzh2JOluuLgPy8vCkF2WGR1fxj5vPsPyqXkEjS0mS2LhxIytXruTtt9/mF7/4RafmK4uLi/nZz35GZmYmgwcP5sUXXwSgsrKSqVOnMmDAAKZOnUpVVZXb/bdt28agQYNIT09n3bp1HR7H+UKIMP1AR39gkk6H5MZydfXKqJ+93Fj0nbmR4F+UrKdiz9p7niB33LWrhT9t+DL4/GnHdXtJgqJvoLrIjx0VVAK7gFIc83TNmlcLDuIMlHBsYNgPQhPt8zeVd39OZDEINbTlWVpaX0oeqC/CtAMVrft0FKfx+LDgTe7R6b0j8HRN+GzTHau2wdP1qSVRd206a9E64PN5QYYIncTMXsV8OBfG9A9eKHd1dTULFiwgPz+frKwsMjMzO92mXq/n2Wef5dChQ3zxxRe8/PLL5Ofns27dOqZMmcKxY8eYMmWKWzK02+3cfffdfPjhh+Tn5/POO++Qn5/f6TGdS4QIk44ToidIkkRBQQHF5eWIBkcsvTurzfWzhJc5TAEl5icQIyZgV6qvtoOWhykETrzu2g/EwlQgWeHoFmgo99CoFmXA/4BTtFmRinXZQhtpBoqTIJa17mvDWejAH6KjdbtiN6SrFU7w1U4lUETgPywFZ2hLJ3GDulyHW7YdvD0ABvKFBkqaTmle3ucjHdt4+1twetB0tkgFdXrFbck+GQRZppeunt8k7GP1kDpsLY0d8mS5w1dffcW1117LjBkz+OMf/4jJ5G8UmnckJiYycuRIAMxmM5mZmZSWlrJ582YWLFgAwIIFC9i0aVO7fffs2UN6ejppaWmEhYUxe/ZsNm/eHJRxnSuE5jCDjKqqKg4dOkRCQgJDRoygUK/3yyvlejEqmqjYAdFBMK7buWtDaxFqr9POzj0q6Igr1c29IuBC1O7u5+q0n+BmpZf2ZKClHr57HzIWQ5i7IEQbDpm3g7RZj8p8pVJDUwnw6UBxT/FrHHOhykFoX34KtdMAgtad6s6t6audAtpyPzuCehyk77p/62d7Zev4/AkmCnQM/hClm21kNxt4thzbN+SuOol2O8/rUH+ABkFmtOEMz4+wkh7Xl+rqaioqKtQ6lNHR0WoOZUREhN8P9ZIk8corr/D+++/z97//nUGDBvm1X0dQWFjI3r17GTduHOXl5Wo0bmJiIhUVFe22Ly0tpXfv3urnlJQUdu/e3WXj6wqECBP/LExBEJAkyWOdOavVypEjR2hubmb48OFERkZSeuaMWkAaPLt5XNeprNeMenMuaAGr5LytN7huE4y0En/a8XhMLhD9cO26tusOHbEwFbRUQuEr0PduCFPiHiQcVuUxHK5GRWxcS5TaPEvl6w3kS/kOR11OT0TnL2HuxxGd6o50lXa0cD1RLTiO0Yb/pO9q0X2Lw+L2oKIkSy67+Du1EAzydLONO6vS3fyjK6m2n7v0/lDo0inIgvpgEiXYmWss4anxsYQbHRJ02jqUdrud2tpaampqOH78OI2NjYSHhxMbG6uKEOj17W/dVVVVLFu2jKSkJLKysggP7yIBZRwRtzNmzOCFF14g2iV9zhPcBVb+2PI/Q4TpJ5TAn7Aw50dlWZYpKyvj+++/p1+/fk4/esFoRGhNNvZkKTn9LTtuMBYr6O0giI6YoXqgohlapMCj/pX+OpP3GKhl6M/8Y2fnAtR2OuJG1MBeCyfXQ/x1EJGGgyzrlcZpT5IKURppI5lAxmADFOvSHbn5SyhVONzEVs0yZT9/K6SU4SDMjt6zqnE81HmArAQg+UIn5+c9tulSxqwtUNVzh47frmc/r/trVyHZtvWSsr2sdO7wBveRGngytoIb+yd4JAudTke3bt3o1q2bo11Zprm5mZqamnZW6IEDB8jIyKChoYH777+fX//618ycObNLichqtTJjxgxuu+02pk+fDjjKfpWVlZGYmEhZWRnx8e1F4VNSUigubhN2LikpISkpqcvG2RUIEaafcEeYjY2N5OfnEx4eztixYzEYDMiyjN1uR5IkwlrTStCkrXh6OpVxKNI0WaDF5nDB6kQQdZBrg3yr48vqKD/4Y2G6I3VXgg/UUvXUZkctXnft+NO/t+WyFRo2AxFgigehOw5CNNDmctW1fg7zc+DuOpZxuHhraS9G7k0dyB1O4LAutbqxSnu+SnAp2x/HfWURf1EN1NBe2b/178bD0HwCwmXn5UGBH65Xp+WC92vHrWWIq0XqvgPZ23atDYcJMpObT/FiskBaXGBCBIIgYDKZMJlM9Orl2FexQjdt2sSGDRs4fvw4F198McePH+fTTz9lwoQJQZu31EKWZW6//XYyMzNZuXKluvzGG2/kzTffZPXq1bz55ptMmzat3b5jxozh2LFjfP/99yQnJ/Puu+/y9ttvB32MXYkQYeKfW0Bb4kuSJL7//nsqKirIyMigW7duTkQJIIoihogI1SXr6f6peF+VYEudDDp72z1apxdoNIRjszUjupCuJ7i6e8E5F97d/u4k+9wRZ2dcoNr2AhFScGelBtKOa3ue1tntIFe3BtFE4lDrCcdBksq9x0D7k+tPB+AgykO0RaRqXbHKPKI/pncjjkCdFjQTuTgbTr6eJBpxCKV3ptDFER/7y46HESGQp7xAA346tK+roo92fhLN3503fUUgWrKxuLaI3wzo6daV2hHodDokSWLv3r2MHz+enJwcysvL+fzzz9m8eTMpKSkMHDgwKH1psWvXLt566y2GDh3KiBEjAHjyySdZvXo1M2fO5NVXXyU1NZX33nsPgJMnT7J48WK2bt2KXq9nw4YNXHXVVdjtdhYtWvSjUxcKEaafUCxMJainV69ejBs3ThU9sNvtyLKMIAgqASuE6c1yUyouaafEFC+g3mRCd8kl1MsyZGUhW61+k6Xr352VolMQrOAhf26inty5Tu34OY6ALHPlKUbJYwy4ATew4pjv04qba61Lf8kSHGkgZ3CeA9WSpj/tnGgdiwH/TqLrNjIOC9NbwJCE7zSkrnDF+ljW/uv0zryuuZrK3+6fmdq+CB2Q3tjEE41WLk/vFVQ36eeff87KlStZu3Yt06dPRxAE+vTpQ58+fZg9e3bQ+nHF5MmTPYq87Nixo92ypKQktm7dqn6+9tprufbaa7tsfF2NEGH6CUEQ+O677wAYMWIEERERyLKMJElIkoQgCO0Cggyt0niuLlkZx724CYexoRClOvVkMGBMTqbplltovuEGEj/+mPKdO9V2A7l3K9t2JuhH219HvZGu8Ec03R8IrRaWJ8vZ03yxX/10liS1KMURkaotzOwqq+fvBPVXOH48CrQn09UocneSbTgCj+x0vIJIPg7C1dF2nlz6stYE2Kbg4d3f/fxZLzg/yLW9nDuV2+2oJc62d+f1ILT+M8kCY8uq+W1YAom9uvt5IL5ht9v5f//v//Hhhx/ywQcfkJaW1uk2Fy1axJYtW4iPj+fAgQMAzJo1iyNHjgCOfM7Y2Fj27dvXbt++fftiNpvR6XTo9Xry8typglw4CBEm3l2ySlDPyZMn6dWrl5r8q3W/aq1KLUS9Hp1mzlMxXKw47lWuwZd6UcQQGUnYlVdy+oYbSLv0UuLj4/nq88+9atI6jdflXft3R75s1z4DmQv1hmARZkeDh9qdH29WZGeJswH4pvXdldgUi9Bf4irE4drVEq/rPKi3tgQc847VBH5cSj8SjmNxox2rRVUWiFaQRTfftee4Gud3X2Px9wfgYw7TU6fe93Eu56X8i7HIzDnZwJIeaV4rjASKM2fOsHTpUgYNGsSOHTswGv0t8Oodv/zlL1m+fDnz589Xl/39739X/161ahUxMTEe9//000/p0SN4mrc/ZIQI0wsaGhrIz88nIiKCfv36qfMPClmKoujTzaI3mRwWJg6SbMZhVeo0Lz1gMBrR9+sH8+YhTJnC6MxMDK2iB3qNa7ej925Jr0cUBbDZHXI33rb1sFxxHfu7r+t4XedU/YG7+VhXBMvq7TIcxOFGVeTjXF/+BvtIOAJ1lKoi7oJ9/CHe73BYqB2931qAo/ic/9Qm7Hv8jvxwofoF13OhfXls17/qJLJmW2+MrpOhX1ULd5XrGJ/YJ6hkuWvXLlatWsXjjz/OtGnTgurevfTSSyksLHS7TpZl/vGPf/C///0vaP39mBEiTNpbmO6CekpLS2lublaVOPwhS3AIsMuCgFUQsMmyak0aaLMqTbGxxN54I7rrr2fQ+PFqOLkCfXi4kyatv3EaMiCLIkRHo7/hBoT9++HgQY/bat9d/1YQiGKQa1vaPgIJp/BGdmIn7hudIlFfJ0pBGQ6SUwTbXcXaBTyTnGu7p1vbc7UutSfTF2HW4VWZxy+U4Ryd64n0tDmYnlzG5xKCV/Z0cct6+ru9S1dAwGQXGVcqscCagGxr5sCBA1itVqKiooiJiSE2NpbIyEiPedyeYLfbee655/jkk0/4z3/+Q58+fQLav7PIyckhISGBAQMGuF0vCAJXXnklgiCwdOlSlixZck7Hd64RIkwXVFZWcvjw4XZBPVFRURQVFVFeXk5MTAzdunUjNjbWp1tk9IoVFGzbxqkvv6SxvBxbfT2SxYJFljEYjfQYPJi4hQtJnDCBtAED3F5QroTpC4rrl7AwwgYNovvvfofp0ksxtQYHePM8up/faYt98RVtC56JUgt/nr399ZAGcgsKRvyOx4a17+Agla90UCvjlMGvfRnwzyq04ZgD1QoVuBKvL6ECcAQL1btZFwiJncSrWAFAU6kjQrZD7QcKX3OfTjypFR7wZd56Du5R1ouIdGsWuP5UFDN6DnG6fmVZpr6+nurqak6cOEF9fT0Gg0FV8ImJiWmX161FRUUFS5YsYejQoezYscPrtl2Fd955hzlz5nhcv2vXLpKSkqioqGDq1KlkZGRw6aWXnsMRnluECLMVVquVw4cPY7FY3Ab1REZGMnbsWOx2OzU1NVRVVVFSUoLFYiE6OprY2FhiY2PbyVgNuP56Blx/PQA1RUVUfPMNpZ99Runnn9Nt3DjirrySIWPHEhUV5XFs2vQUBa4eOaflooihZ0+633orSatWoW9NIlbcw+7gyWDqDDl5a78DanIe3bPnTBA5kLBduw75UCxCRS3Yrc7rtE8d/s5d1uOwVBXrUnHBKu2KeCYw7Vf+PT7JzitqcQgm+HBz1B0BW4Pz4Tn98jwZe50lVg/7Oz/PtPffal2zzr+xNhbWLhMAvSySVKljoWUQgxLaJ+ALgoDZbMZsNquScBaLherqapVEbTYbZrOZmJgYIiMjiY6ORqfTkZ2dza9+9St+97vfccMNN5wXRRybzca//vUvvvrqK4/bKMID8fHx3HzzzezZsydEmBc6ZFnm22+/JTEx0Skx2F1Qj06nIy4ujrg4R1UBSZKoq6ujurqaY8eO0dTUREREhGqBms1mdd+Y1FRiUlPpNn48xmPH6N27N8nJyb7nQcPD1TlMCYex0UjbHKjq5RMERKORmDFjGLBmDTGTJzu1bYiI8H0ufKwTdDrQichWG55LUfhu3x9L1df42lxige3XkfX+Q0DWRyHVXox46FtosDpbgCJtqRjeame64lscX7qyr6s15Q/xluNwp2oZIVCysuBs5XqC3Y+2vLln/eUHf6xKH41786y0/0ocgw636RlRamZuzESiY31fVwrCwsKIj49XlXAkSVKt0C1btvDMM89gMpmoq6vjiSee4PLLLz9v8nGffPIJGRkZpKSkuF3f0NCAJEmYzWYaGhr4+OOPWbt27Tke5blFiDBxEM2oUaOQZdljTqUniKKoulf69OmDLMs0NDRQXV1NUVERdXV1GI1GdQ7j1KlTCILAyJEj/Y5y05tMoNMhCQLNsoyV9nmbelEkKiWFtHnzGHDvvejdWKz6TmhLyoKAEBtL3c9+hvXzz6G8PGDC1G4dzFtAR4Tcva3v+ED0SOY+2HouwvCPvyHU1jo/IUAbcQbiXTsLlNAmg+duis2fwSuu1M7MI37VOg7lzuHBHSoHKogQrB+El3Zkj5VK2nZ0P53QFvAjIBDbEMZlZwZwRc+RnSYzURSJjo4mOjqaqVOn8t5775GZmcmkSZP48ssv+cMf/sCAAQN4/fXXO9WPN8yZM4esrCzOnDlDSkoKjz32GLfffjvvvvtuO3esVoigvLycm2++GXBYo3PnzuXqq6/usnH+EBAiTA38SRXxBUEQiIqKIioqSn0ya2pq4vvvv6eoqAi9Xo/RaKSoqEh14yrRsJ4Q3bs3aVdfTVleHs2nT2NtaECyWBAlCUGWCTeZSLjiCi5+8EF6jBnjsR1DZKRHl6w7qDcMoxExI4OK6dPpMWsWkTNm0FJR4fMe7a6+poJA0krc9aOOTe5c4I/HPgJxvwpAWBT2tOuw9bsDwwurEQqOgmx3zrXUWpr+umIFHDmPdThbl9q+/XlikHAE+1gJXJBYQTNtOVFedBqt9VBX4Hm9T/jzfbojfXef27XljTS9D0gv6+h+Ooab5J+RGh9cDdSsrCxWr17NunXruOaaaxAEQRUg8CQUECy88847bpe/8cYb7ZZphQjS0tLYv39/Vw7tB4cQYeKYV1iwYAGjR49m0qRJDB48OOBoNk9oaGjg8OHDREZGMnnyZPR6PVarVZ3HKCwsRJIkoqOjVTeua5UBc3IyV7/yCrIsU11QQMX+/RTn5lK2Zw9SUxNDFyzg4uXL281zusIQGel1vas8HqKILjoabrqJmltvJWPcOEwmE3ovlrE/aSDQei/zwzLypy1f35QnwnW7rT9EqVkniyJSTD9sl61GSppC2NN3Ix79Fqw2ZzLTlgYLZP7wNMhlIkKL5BxerHXx+qvsU0nnrvjTONJjXNtwcevaW6DpFBi8WZnBdDF4Ik8P7mbnhzF3gT1tOyv/h1vD6VOaxjVxVxJhCJ5Gq81m4+mnn+azzz5j69atbt2fgTy4uxMhePTRR/nTn/5Ez549AYeUnTu1nW3btnHvvfdit9tZvHgxq1ev7uBRXbgIESYO2bvVq1eTnZ3Nc889x6FDh+jduzcTJ05k4sSJjBw5MuAINUmSKCws5PTp02RkZDgl/hoMBnr27Kn+gJVAourqak6ePElLSwtms5nY2Fi6deumBhIJgkC3/v3p1r8/g1qrBDRVVmKK869Kuy/ChDbSFMPDMWVmYp07l+Rp00hKSlIvXDEszGfwkOucULtt9DpEQXAIuErtWckTUQYr6Md13ipgCCAbIqhPvYzjgxZRWxHO4P+7g/Bvc0CytCc2ZaCBkKXOgPU7M7raOgS7pParvivzoP7mXlrxHG3lzz25FM8nS0NQspsvSWj3R4B9dwZu2ndHmu3XCwiIRNWaGVV1GRfHjwrqfOKpU6dYvHgxEyZMYPv27UHRmXUnQgBw//3388ADD3jcz263c/fdd7N9+3ZSUlIYM2YMN954IxdddFGnx3QhIUSYOOYRhg8fzvDhw7nnnnuQJImCggKys7N56623WLlyJbGxsUyYMIFJkyYxduxYIiMjPV481dXVHDlyhPj4eMaMGePTWnUXSKQEAij18CIiIlQXrtlsVtv0lyyhNXhIEBxk58HNI+p0hMXFETltGqZbbuGisWPbzbV6szDdwaknQYDISJomTMBy5AhySYnrFp739bBe8XR2ZG7S7TpfO4g6pG4p2Kbeh37SEjLOlGN44j70eblgtThbfwIaKSc/2gccgUNmKupG070oH7HF5i6w0wFPV7B2+zocka2+omN9cUEJjogzXwStqZju5EEOtlXp7d3bPl5WaiNhdZKeuFO9uUI/kx7xCR0bpxvIssynn37Kr3/9a5555hk1jzEY8CZC4A179uwhPT1dldqbPXs2mzdvDhGmC0KE6QaiKJKenk56ejqLFi1ClmVOnTpFdnY227Zt4/HHH0cQBMaPH69aoXFxcVRVVbF9+3bS09MZOnQoEX5EpXrqXwkESE1NRZZlGhsbqa6upqSkhLq6OgwGg2qBxsTE+KUqEt27N9F9+tB09qxjHtRmQ5Ykh/KPIKALDydm6FBif/lLBl55Jb1aa3u6Qm80+hVn0i6AQqdDN3Ag0StXEjljBpFTptBYWuqXhefNNXvOYggFkMNMSEOuwHrLo8iJFyEc2EfY4w8g5n8FNouz1qHWwjTi+2oTAFFPS0RvDsffyaCvv0FfnQtW2bkt5W9/c3NO4EhLkXFUYekICnDI4WnH6gGlOQ63rBOv+hMNG+gXGcD2nmphulqaQut6fYuJbkcvJvZMOkVhpdTE1KsFnDtjCdpsNp544gny8vLYtm3bOasHuWHDBv7yl78wevRonn322XbiKKWlpWrqCzhqV+7evfucjO3HhBBh+gFBEEhMTGTWrFnMmjULWZapqalh165dZGdns2HDBkpLS7FarUybNo2JEycGtRadIAhERkYSGRlJcnIyAC0tLVRXV1NRUcGxY8cQBEG1QGNjY926kDNmzGDgTTdRefQoFd98Q3FuLqe+/JL68nL0BgNJ06bRe/p0Bo8e7TUQSWc0+gweauc2jY4m+qqrSHzsMQytaiU6H8FO/sDfnE5/3a4e4ytEETmmJ9ZZq7FfOg9Jb0T44wYMf34JTpchS3YE18Aexar0x5svgGwwU959ItUXP8jA5maMe56DZovTNuq9XiFmX2jGUVJMqWxi87KtJ8g4yFIbHesF1kZHvJMKX9afayBTR+GpHy+BP9rPDrrUE1bTi6G1vyA9fRikoxZvPnPmDN99951avFkhUJPJ5JeFqESYXnrppXz00UdBK/XlC3fddRePPPIIgiDwyCOPsGrVKl577TWnbdwFFp2vdJYfMkKE2QEo5HTdddepRVtTUlK49dZbOXDgAPfccw+nTp1i8ODBTJw4kUmTJjFw4MCgBRIBGI1GEhISSEhwuIqsVqs6D3rixAnsdrtTIJFC4KJOR4/MTHpkZnJRK/kf/+orio4fZ/jPf+6XiLIugPQUwWAgcuBAUu+5h/h585wuQp2XuVAF3uYaZQCdiKgTka12t2znLr/O3XqAEjuUNcEAoLeJthJkYeHYh4zHetd6bDHJWDZvwfbKH9B9sxeL3YpOBJ0O9CIIrX8LAgh+6gjKoo6myGQO976F2j5XE28zIr/3AkLVWWf9WQicWEpwWJdKO9U4TL9AnueswNd4FitwHY8cwBC74p7sJdin7eXKriKCZMR4cjijTEuJ7tE21REeHk54eLh6rSnFm2tqajh69Kiae61I4CnVO9R+ZZmHMt5FAAAgAElEQVTt27ezdu1annvuOX7+8593wUF7hjJugDvuuIPrW4VUtEhJSaG4uFj9XFJScs6s3x8TQoTZSRw7dowlS5Zw1VVXATC9NRjHZrOxd+9esrOz+e1vf8uxY8fo378/EyZMYPLkyQwbNiyoT5gGg4EePXqohCdJkkqghw8fprm5maioKNWNGxkZSXNzM4cOHcIUGcllt9zi93gMPqxnGUAQ0EdFkTxjBukPPkhE377tthM7IPXl5EILC6MxNZVaUzgcOgw2/8wnT6RZJcMeC3xmAWocF0ekTqDfgET6VCYjL7gb4fhxdNXV6O12VThfDxgE0Ak4yFMEUzgYfD0fCSDrTZxNnkD5+HsYOGoKdrsd23t/IuyrHFDmLtvf2/2zLq04AnUUsXZwkF6Tm5PgDadxEK4f+1ibQLI6LxNcP/iaZ/SHRH1t49ezqWJZiggtMcSWzWBYz5l+xRx069ZNdWsqUyY1NTWcPHmS2tpaZFnmr3/9K2PGjOHgwYMcPXqUjz/+WBVGOZcoKysjsXV65YMPPmDIkCHtthkzZgzHjh3j+++/Jzk5mXfffZe33377XA/1B48QYXYSl112mdvler2eMWPGMGbMGFatWoUkSRw5coSdO3fyyiuv8O233xIfH8/EiROZMGECY8aMCaobVxTFdhe1EkhUUFBAdXU1NpuNxMREEhMTA7J+w6KjCYuObpsHtTtbd6LBgLl/fy5etYp+c+d6bEdnMPg1F6qFTGsCemwsDVOnIi1eTNyf/8zZw0c6JTzgTrjGBtTYZfYd/p59h78nAugOxAFmHEZaGK2EKUOYDOESROi9uHbVuUgdNnNPCoZNJ+LG1aR1d9RM1JcVYd7xAWJtg4OktGkj2vlLf9BkghMStFgc5rKiMqSU5lLG4wsFOEjWD8dCbQnUlYLR1crswvlJj9uKLusFZzECAEEOw17Znz7NK0lKaE8kfnWvmTJRrLKmpiYOHTrE3/72N8rLy4mKiuLBBx9k4sSJzJ8/n0g/ItY7AnciBFlZWezbtw9BEOjbty9//OMfAWcRAr1ez4YNG7jqqquw2+0sWrSIwYMHd8kYf8wQfCTFdm3G7E8YsixTXFxMdnY2OTk55OXlYTKZ1ECi8ePHExMTE/R5hIaGBg4dOoTZbCY5OZna2lqqqqqoq6tDr9c7BRJ5sjjtFgtnDx+mfN8+inNyOLV3L01nzmCtr0dvMDB4xgxGP/QQZk0QgTtsmjOHgg8/RLJa260TcdyfI3EQU1Tr56iwMCL796dx8WL63XwzPRIS2HbXXeS/847bdoTW/SJaX0p7kZrPSj8n9JBlaxPUcUWYpp2o1neTph3lc6QeIk0QFt7auKn1pfwdHU512nCKpyynzyU3OM0XG9Y/jP6fr0JTQ9sBaF+61naMmg61/UQAJhE5vjfyVz0Q9x0Ea7NzWyIO1s9sPRDlwJR2tagFtgBVre0bNX1GtH6ObPt8phCObwKjRXOu9BAVAQZv+2qXK+fLmxWttBWh2U+7v/bcmMBugqYIkRaDkSbBRAsmmmw9aDw5hf7m+4kI9z/a3BdkWWbbtm089thjvPDCC1xxxRXY7XYOHDjArl27WLhwYVAfjkPoEri98YYszPMEQRBITU1l3rx5zJs3D1mWOXv2LLm5uezcuZNnnnkGm82miilMnDiRhISEDhOoJElqtRVtXmhUVJT6VGyxWKiqqlKDGwB1XqZbt25qIJEuLIz4YcOIHzaMofPnOwkqCHo96ddei+iHe1fnwyXr9LQmiuhjY5F/9jNYsoSLx49XCV3sQPCQW+EevR5kCeyeM+69zYn6zOsUBCRzHAVjb0J/8wrS+6Y7rRZ3Z6P7cDPUNbaZvK5WpTdHgACSGEZVtxHUdLuF3iUbES2WNoUgtSMc85r+oJGAik3LLumi7sboVwpIsKCdwxQcnQvosTUmQ8VShsTPCupDqcVi4dFHH+Xw4cNs375dnT/U6XRq6loIP16ECPMHAkEQ6NGjBzfddBM33XQTAHV1dXzxxRdkZ2fz6quvUlVVxfDhw9V80H79+vnlSq2rq+PQoUN0797da15oWFiYUyCRzWZTK7MUFxdjs9mcKrMo0YFaQYVAoPMjn1PGQWSmwYPh1lvpP28ePVoFH9R2fBCmr7RKWRCwR0ZQP2ok8pdfQWOjlz0C7wMAfRgNqQM4On0VfafcQLhrylFDPYbf/38IpWVg0/gzlcAhX0WmBQE5qif2y+7EOHkFPV58Et2pk2B1I3gg48jJdIWi7K91A+8CWvBbyq/iGE46sn5RUUciZP1JU3G3UjZSVzGKSOkxeiQE1+VYVFTE4sWLueaaa3j22WeDUkDanXLPgw8+yH/+8x/CwsLo378/r7/+OrGxse327du3rxqApNfrycvL6/R4fuoIEeYPGGazmalTpzJ16lTA8fT65ZdfkpOTw+rVqzlx4gQZGRlqJO5FF13kdJHa7XYKCwuprKzkoosu8lpCzB30ej3du3ene+v8miRJ1NbWqsIMzc3NREZGqpG4UVFRAT2tG8LDEfV6ZEly5IO6TA8IgkBYt26YL7uMuLvu4qKxY92mu4gGg89oWy2cejEaMQwaROTjj2MqL4e8rwNvwxtEASmmG8Xjp2BdsIpBGUPanyOrFd1LzyAc+NYx3whtN36tfqw7SToB0IdhTx2ObcbjSAMuRVdRRkT2dsSGZuf9vc2BatNOFMKsbX25M7g9pG5UFYFgcxWr8GN/7fEEAn/yOFuXS7YYKkpuIzHu1xjDzAF25BmyLPPf//6X3/3ud7z00kse4xo6AnfKPVOnTuWpp55Cr9fz0EMP8dRTT/H000+73f/TTz/1K/I9BP8QIswfEcLCwpg0aRKTJk1i9erV6rxIdnY2zz77LIcOHSIlJYVJkyYRExPDxo0b+fOf/8yoUaOCktIiiqJqXfbt21etzFJVVUVhYSH19fWEh4erLtzo6Giv/V581130HDaM4pwcKvbvp6myUg0kEgWB6EGDiJs+ndG33+71ovfl2gU37lJBQIiOxjxnDj0eeghdfDwRmzb51OP1GwKg19OUNpDjN/+CxNm3ewz0kD75COGtN7DVN7WWaVOH6MjtdEeWrZ3IpmjsV8zHeuNDEOkI8NL9bxvid8fBorEutRam1vCRcQQXNeOsTKTDER17unW534msHpYHI4gnULS2IyNS19CXE0W/ITnpOsIMwQu4aWlpYe3atRQUFLBjxw5V7jJYcKfcc+WVV6p/jx8/nvfffz+ofYbgGSHC/BFDOy+iSPodPHiQBx54gPz8fJKTk3nooYcYP348kydP9inpFyi0lVkUlZCmpiZVE/fw4cNqIJHy0gYSKfmgIxYvRrLbVUGFE9nZ1NbXM2DxYoaOH++zmkug6SmiwUB4//70eeYZzJdcgtDavqjTBedeLQhIUVGUXXY5NcseIG3ESI8PDvb9+7E/9SS2ikoEO4itaSmC4KjCIoqg07vhEL0eObEPliXPIg27AsRWFpQkDBvWOwQPtJGw7ghToo0oFZev0PbZvltEtMkIYZ6EYV2OxUY7SbwOwdeO2iAo3LxrIGGivGoSTWG/I8xg5tix4zQ1NWEymdTfpK8HO08oLCxk8eLFTJs2jRdffDGoedb+4rXXXmPWrFlu1wmCoMruLV26lCVLlpzj0V14CBHmBQS73c6SJUuYP38+H374IYIguJX0GzdunBpIFBcXF9SgB5PJhMlkUvO+lArzlZWVFBQUIMsyMTExqhtX0alVBBXscXHUpKYyOj3d76d1vxWDBAFDdDQJ06aRvnYtRhfpP0EU/XLtehM/kHU6WlKSKZ47h4S776WPRnTfFfbiYhofegi+zUe0SSpJWnEQpdCa12nSabhAAEyR2Cddg3Xpk8g9kp3aFF/9A1LFaQSrvY1H3AUN2XFUL4nGueqJCBhEWuw9sLfYMcmVPs+Hgu+/gpYGv7JP3CMY7liVTEWsUhyldb+ke9/fEKv5jciy3O7BTqfTqco9npSytPv/+9//Zt26dbz88stMnjw5wIEHB0888QR6vZ7bbrvN7fpdu3aRlJRERUUFU6dOJSMjg0svvfQcj/LCQogwLyAYDAZ27NjhpGHrS9KvoaGBkSNHqoIKKSkpQSVQ1wrzSmWWqqoqSkpKsFqtmM1moqKiOHv2LAaDgdE+pPlcIRoMPkXlBZ0Oc0YGQ1evJvGGG9xG1oqKRE8HIURGUjtuHLV33IHQpw/5+fmEh4erDwdaS8Z++jTVDz6INS8P0WJVDT+Vt1rJUtTet0UROTEV68JV2K+5DcKcg6bkinJsf38Xub5ZTbtUTotyWGrtUDuO+pipOAJ6WsUQZKORqvjhNDKJhOpXENxUknEcrMs7jmAfWW632O1n3yv8hFvr0kCNbQA1kb8lvt9V7X7PgiAQERFBRESEGiGulNyrqamhqKgIm82G2WxWCTQiIgJRFGlpaWHNmjUUFxezY8eO8zY/+Oabb7JlyxZ27Njh8XpVji0+Pp6bb76ZPXv2hAizkwgR5gUGb4LvWkm/6667DnC4UHfv3k12dvY5kfRzV5mlqKiIwsJCwsPDaWlpIT8/XyUZs9nsk8B7jRxJv6uu4vSBA7RUV2NtbFQFFQTAEBVFn2uuYdxvf0ukm3qD6vkRxYDu3wqVCDoddO/OmYUL6X3HHaS1Wq6yLNPc3KxaMkeOHEGn0xETFkbYE08gb9+O0NLSTn5WAEQZTFqXapgR+8TLsd7/W+R099Gd9jffQDp6DHuL47gVohRbXzqdS1aKBUdQTxQgCtgjYyjMuJmomx/GfNttYLX6TGNxOh/eynoFCn/csm6WSUIkJ20/Jzz9aXrGeM8D1sK15J4kSdTV1VFdXc13333HHXfcQVRUFFVVVUyZMoV33323y8QHfGHbtm08/fTT7Ny50+P13tDQgCRJmM1mGhoa+Pjjj1m7du05HumFh5BwQQhOsNls7Nu3j507d5Kbm+sk6Tdp0iSGDRsWkPXnDRaLhSNHjgAwaNAgwsLCVJmxqqoqqqurqaurw2g0OgUSeQrXt1ssnDl0yFFgOyeH8r17EUSR0XfdxeAFC3wG9BTu2MF/5s+npbra7fow2gsVRJhMRAwfjmHlSjKuvNLnuWkoLKT4gQewfvIJotWKnvYeUYHWnH4BTBEiEck9sC9YjG35Q+Ahv1U6epTmXy6A/IOIrcrnWutSEMAY1mqUuogEyLF6Gnv1peim35By6TQ4cABp7kwi6k5hMMjexQZaBRRaJPj6E2goBpPs4GBFcMJkAHME6BWxBaWtMNqUJFyFC8LxTNaCZpvItpPVHNGL0m7LSRhyJzpDhx3D7SDLMv/617/YsGED11xzDadPnyYvLw+9Xs+bb75Jenq670Y6CK1yT0JCAo899hhPPfUULS0tavT6+PHjeeWVV5yUewoKCrj55psBxzU9d+5c1qxZ02XjvADh9pEtRJgheIUi6acoEn3zzTckJCR0WtKvoqKC7777jrS0NCdxaHdQrLSqqipqa2udonVjY2M9kpQsSTSeOUNkqzvYF05kZfHvefMchOnmulA0yyOBSFEkumdPDNOmkX733ST6cdOsysmhaO1amvfuRbRa2xGlEndjpFXVKNyAaeRw9I8+jjzJe6pC3f33Y/vbXxEbGtu5YgXBcfVHRrgQZiRIsWbKhk/G/svH6DlgMLIs07J+PTzzBFE6q291HqPjxDQ0Qs4W0DW1nh/8IEytWlFHCLP16UU2GaiMGkrzRU8T12eiz+8hEDQ1NfHwww9z+vRpXn31VdUzAlBbW0t4eHjAxeVD+FEgRJidwbZt27j33nux2+0sXryY1atXn+8hnRe4k/QLDw9nwoQJfkn6Wa1Wjhw5giRJZGRkdOhmo8w3KS9Jkpwqs4QHUE1Fi+KcHDbNmeOTMM1hYcQOHkz3e+5h+E03tSuw7QpbQwMFGzZQ+uc/YysrQ5RlVT9dmbdUhNzDgChBwBgZif26azk5azaG+HiniE5XC7slJ4fqJUugpBhBkp3cu2ocj6FVrk8hqQiRltQUCq+aR/wv7leFFOwnT1Jz5ZWYTp7AbJS8E6ZS41OE+gbI3Q66lvaEGWFwSOPpXQkyCIRpN5s5GXc9kZc+iynSc4BVR6AUVpgzZw4rVqzo9NSEOxGCyspKZs2aRWFhIX379uUf//hHu1qVELr/nAeECLOjsNvtDBw4kO3bt5OSksKYMWN45513QtXIwUnSLzs7my+++AKr1epW0m/btm1ER0eTlpYW1KoNSiCRQqAtLS2YzWbVjRsREeFXINOpr79my8KFNJ09i62xEcludxKWDxMEYnr0oPuUKQx54AH6ZGR4bdfW1ETJ9u0c+v3vqTtwANFqVYlRW2daWWYCIvR6YlJT6fn732O88koEnU6N6Kyurm5nYcdERFA1fTqWPXsQLBanYFhtWmWkoU3fVo41cnbYSKrvWE3SpClOx1D/yis0Pv44kY01xJha9V+11qBCmAbnAzh5Cr7ZCzqLY7Wq/UsAhGlyWedJKEcAInU0mXtRftEqeo69I6jz7LIs8/777/P888+zceNGxo4dG5R2s7OziYqKYv78+Sph/upXvyIuLo7Vq1ezbt06qqqq2okQhO4/5wVuL+xQ0I8f2LNnD+np6aSlpQEwe/ZsNm/eHPrBgl+SfqdPn8ZgMBAWFsbzzz+vRswGC+4CiZTKLMePH6exsZGIiAiVZMxms9sbbK+RI/nl7t2cPniQ8r172wQVqqqwNTQQM3AgSbfdxqSFC70GVwHs+9OfOPjaa9QeOQItLejBqRyYligVY0qMjMR81VX0evJJDBrhetdUHcXCrqqs5OQTTxCRl4eulSzdFTZReUcUscfEcOLa6USuepTkOGfBcXtlJc1ZWVjr67HgRa2nhbZi0q3MfKIQLBb3KSUBB/4oEn3aai2aaFhZZ+RM9MXYrniOhD7B1WZtamrioYceorq6mk8//dSttddRuBMh2Lx5M1lZWQAsWLCAyy+/vB1hhu4/PxyECNMPlJaWqon54Ci2unv37vM4oh82tJJ+H330Eb/61a+YNm0aRqORJ598khMnTjBw4EBVtchV0q+zEEWR6OhooqOjSU1NVQOJqqurKSkpoa6uDoPB4FSZRelfHx5O4qhRJI4apQoqlOzbx/6PPmLA1Vcz6OKL/bJWv3njDU4fPKhG6ipkprhhFaIMB8JFEWNKCim/+hXxs2cj+pgTViI6TUePUvTvf9PS0uKUkiJo3pULXAgLo6F/GhV330/yLXPcnm/r4cM07dqFbLfTJLgQphIBa6WtGLXGhJVbS7y4VarzR75OgaI6ZAF3fmuLMZqCpGk0DrudbsYEmpubO+yCd8WRI0dYsmQJCxYsYNmyZedEiKC8vFx9EEpMTKSioqLdNqH7zw8HIcL0A+7c1sEuu3UhwmazsWXLFrZt26beFACPkn5KKsvFF1/sc14wEGjrFSYnOxL9W1paqK6upqKigmPHjqkpN8pLidgtLinhlM3G5ffcg9nsv/6oNiJX4Rqttrmx9aULDyduwgQG//73RAVgMbScPEnJb39L89mzCLKscpiWnA2AXhCwm0yUXn4ZzfesIHHIUI9tVq5bh7WuzqHPLoNNaq32pcjnNdKexESwCgJ2nQEEq5dCoG6gvYQUQXgXy1XtRy9S3y2NM5c+TPLYGdTW1akFm1taWtTi6B3RNJZlmXfffZcNGzaoUpI/JITuPz8chAjTD6SkpFBcXKx+LikpUZOCQ/AMvV7PSy+91G65O0m/goICsrOzeeutt1i1ahUxMTFdJukHYDQanSqzWK1WdR70xIkTWK1WbDYbUVFRDBkyJOCcO9FgQNTrkaCdsLyM44YXnpDAgF/+kotWrkQfQPuWigqO3n8/tV9+iShJCILgCCQSBITWgCIBx3mWEhM5feedpMybR5PdTnl5OUePHkUURTUpPzY2lpacHOrz8x06vjg4qsYKJiOILbQxvXbiVQDCwyjvnkZFUwl62V0JFA/QfpUSDrJUWN9Kmy9ZBDncSHnyJJj1LAm9BwKoLvh+/fo5FUc/ceKEUyqSot7jyYPR2NjIAw88QFNTE1lZWWrZu3OFhIQEysrKSExMpKyszO10Rej+88NBKOjHD9hsNgYOHMiOHTtITk5mzJgxvP3226GK5F0EWZZVSb+cnBz27NmjSvpNnDiRiRMn0r179y55ypZlmbKyMgoLC0lJScFut1NdXU1zc7NqxXTr1s0ngZfl5XHyyy8pzs5uJ6igEwRSRo5kwm9+Q/IVVwQ0PltdHYcffpgzmzYhNTU5CFKWEVpdvzpAlGXCw8LQjRhB9OOPkzpxYruxah8QqqurER55BMMXX6iBQ4o716QHsx66h4NR6/fVgxxtxn7rfIoGjuPru+7GUFen5qhq8zAjwhxBPzol71IbcRtGGwlr/datEn3WqFhKx/2CuNseCyj/V0lFUgKlwFHbNSYmBovFQmpqKocOHWLJkiUsXryYpUuXnhMXbGFhIddff71Tqa7u3burQT+VlZX8/ve/d9ondP85LwhFyXYGW7du5b777sNut7No0aJQEvA5hCLp99lnn7Fz504+++yzLpH0s1gsHDp0CL1ez6BBg5yE4rVWTFVVFQ0NDX4LeNstFs4ePkz5/v2U5OYSFhXFxF//GlNr4rm/sDU28s2aNZzeuhW9Tofc1AQWi6Mmqc0GNhuCJGGMjUV/440MevhhYnzkuALU7NhB4R13YK+oQJBlJ9eu8ooRIMkERj2IBhEhtQ/WR3+PdNlUijZv5tu778ZQX+9EmBGt7yZXwlSiYBXXq1YXUHH3hhmoTUijevZj9Jh8fae/W6W2a1FREffddx9nzpyhoaGBO+64g7lz55KZmdnlhOlOhOCmm25i5syZFBUVkZqaynvvvUdcXJyTCAGE7j/nASHCDOHCgVbSLzc3l7KyMoYMGaJaoIMGDQroBnj69GmOHz9O//79/Yri1Qp4K1aMEkikuAH1HlR5OoK64mIOPPMMtQcPIggCcmMjssWCThSRqqsdlVYsFsSEBOKWLWPoHPeBPa6Qmps5vGABtR99hGi1uiVLHQ6eMwsQFWFEuGQSxvUvIPZzFAz/cOxYrIcPY7LbVWEHJ8I0QpRJQ5gG2ufWaIhTMkVQNuIKwu5+mqikvkE7hwD19fWsWrUKm83GsmXL2L9/P7m5ueTn5/Ppp5+q6jkh/OQRIswLBcXFxcyfP59Tp04hiiJLlizh3nvvPd/DOq9QJP0UN66/kn42m42jR49isVi46KKLOqXaYrFYVEm/mpoaAKfKLB1tu3D7do7+/e/Y6uoQZdlhSba6du0NDYiArbkZoW9fhj/2GIn9+/vd9sk336Tg17/GXl3tFDDkKtMXJQiYevbEMG8elTOmU2NxhMXGxsZyeOZM5OPHiZBllTDNaFyyRkcOqC4MZxdsO3UFgZYeCZTefCfxc5ejCw9cQcob8vPzWbp0KUuXLmXx4sVOD1SyLIcCaULQIkSYFwrKysooKytj5MiR1NXVMWrUKDZt2hTKy9JAK+mXm5vL/v3720n65ebm8uWXX7JgwQKSkpKCfsNU3IAKidpsNqKjo9V50PDwcJ99njl0iH0bN9JQXg6t85SiILSlJkoSzRYLMRMnMuneewOKLm4sLGTf3Lk0HjzoaFsTMKS8DECkwUC3YcPo8eijmDRzrjabjZPffEPe3LnoSkqIkGWHiBAawhQcc5gR4Y6SnU5FsdUIWJCNBmrSM6n71Xp6jJnk9zH4A1mWeeutt/jTn/7E66+/zrBhw4LaPjhSUrR1KQsKCnj88ce577771GVZWVlMmzaNfv36ATB9+vSgCqKXlJSwa9cuj/UxQwgIIeGCCwWJiYlqmobZbCYzM5PS0tIQYWogiiKZmZlkZmaydOlSJ0m/f/7znyxsFR+4+uqr+fbbb4mMjPQq6dcR6PV6unfvrrr5JEmitraW6upqDh8+THNzM5GRkaoF6i4dokdmJj9//nlszc2qoMKZ/HwaKyqwNjVhj45m6IoVZEyaFNDYLVVVHPzNb6grLASjEdlqdQQOCQKCJKnBRMbISLrNnEniI4+gd5kP1ev1NOblYa2oQN/64N1uBDLY7NDQ5KiWohNbq6YIrQQqgmSM5uTkK4lc+ww9egZX1KKuro77778fnU7Hzp07iYqKCmr7CgYNGsS+ffsAR9pUcnKyKn6uxSWXXMKWLVu6ZAw7duwgPz8/RJhdiBBh/shRWFjI3r17GTdu3Pkeyg8agiCQmprKqFGj2LBhAytXrmT+/Pl8/vnnZGdns379eiwWC6NHj2by5MlOkn7BglbSrm/fvsiyTENDA1VVVRQWFlJfX094eLhTZRbFbagVVJBlmaITJ/hu924uvuIKuvlZaFuL46+/Tk1pKVJCAjpAqq1Fam5GsNsRJQnsdow9etDnoYfotWiRx3ZcU2ac1uHQILDaHDcanVJmTHQUyNaF6bClJFF1xzLilywPetDNt99+y7Jly7j77rtZuHDhOXO57tixg/79+9OnT59z0h9Abm4uK1euJDY2lo8++ogPPvhAtWRDCB5CLtkfMerr67nssstYs2YN06dPP9/D+VHgwIEDyLLM0KHtE/i1kn67du2isrKSYcOGqYIK/fr16/JISiWQSKnMotfrnQQVJEni4MGDmEwmBgwY0CGFpMItW2g4cQJrZSVNp04hNTRgb2zEWlWF5cwZdKKIOSWFwb/7HdFuzpMCS20t/5s3j+qsLCI0AT/KuxLw6i4IVjSG0TB8OOVL78R88cXq8QVDsEKSJN544w31NWTIkE63GQgWLVrEyJEjWb58udPyrKwsZsyYQUpKCklJSaxfvz6oqSFXX30169evP+fHe4EiNId5IcFqtXL99ddz1VVXsXLlyvM9nAsSFouFL7/8kpycHHJzc1VJv4kTJzJ58uSgS/p5GoMSiXvmzBmampqIi79kN7IAABSOSURBVIsjKSkpYIKRJIm6oiJsTU1YampoqapC1Oup//57pJYWpKYmmkpLierblwGLFhHmI4m/6fRpPpg4EU6eVPMulepdWqEerTCQXhAQoqJonjmT1McfxxAV5ZQParVaVeH82NhYv4XzFdTW1rJixQoiIiJ4+eWXz3mRZ4vFQlJSEgcPHmxXtk4Rzo+KimLr1q3ce++9HDt2LGh99+vXj6NHjwatXu1PHCHCvFAgyzILFiwgLi6OF1544XwP5ycDu93OwYMH2blzJzk5OV0u6aft99ixYzQ1NZGRkUFTU5MaSBQIwdhaWrC3tCDZbA4JPL0eS20tsiQhW600njpFbHo65n79EP14ECjJymLnwoUIFRVqdRJtNRZFcF4lTr0eISkJ029+Q/+5c92OU5Ik6urqVAL1VzgfYP/+/Sxbtoz77ruP+fPnn5eo182bN/Pyyy/z8ccf+9y2b9++5OXl0aNHj073e/bsWaZMmaLOo4bQaYQI80JBbm4ul1xyCUOHDlVvHk8++STXXnvteR7ZTwuSJPH999+rBLpv3z5V0m/SpEmMGzeu05J+dXV15Ofnk5SU5FacQUswVVVVNDU1ERERoQYSmc1mj/1LNhvWxkZkWcZSV0dEz57oAyD8rHvv5bvXX8dktxOOM0G6FsXWh4ejGzuWtPXriQ/ADanNd62qqqKurk51U0uSRM+ePYmJieHVV1/lb3/7G2+88cZ5DX6bPXs2V111FQsXLmy37tSpU+q8+J49e7jllls4ceJEUIg9Ly+PtWvXqkIHIXQaIcIMoetgt9sZPXo0ycnJXRYF+EOHLMuUl5eTnZ1NdnZ2pyT9lKjesrIyBg8e7Hd0p1KZRbFAtbqqSiBRMNzILbW1ZK1YQckHH2DQ6M/qXF+CQFj37kTMns3Ihx/GFBvb6b4VN/WmTZvYuHEj9fX1mM1mHn74YaZMmeIk9H8u0djYSO/evSkoKFA1aV955RUA7rzzTjZs2MAf/vAH9Ho9JpOJ5557jokTJwal7/r6eq644goaGxvZuHFj0Nr9CSNEmCF0HZ577jny8vKora39yRKmK1wl/Xbt2kVjY6NPSb+Wlhby8/OJiIggPT290wSn6KoqgUTaaN3Y2NgOzXmV793Lf2bNwl5V5ZDTs9kQJckhAi9JDtesKGIaOJABa9eSeeONQXeRfv311yxfvpzly5fTr18/du3aRW5uLqmpqfz5z38Oal8h/OQQIswQugYlJSUsWLCANWvW8Nxzz4UI0wsUSb+cnBxycnI4deoUgwcPVi3Q/fv38/nnn/PII48EZW7LHZQC1MpLkiSnyiX+1Je0W62c/vZbKvbtozQ3l9P79tFy9qxDEF6S0On1xEyYwOXPP0+Pvn2DOn5Jkti4cSN///vfefPNN8nIyAhq+yGEQIgwQ+gq3HLLLTz88MPU1dWxfv36EGEGAEXSb8eOHWzcuFFNefEl6RdM2O12p0jVlpYWNZCoW7duPiNVZVlGttupOnaM8v37yf/vf4nq358pa9YEfezV1dXcfffdJCQk8Pzzz2PyUWy7I+jbty9msxmdToderycvL89pvSzL3HvvvWzdupWIiAjeeOMNRo4cGfRxhHBeEVL6CSH42LJlC/Hx8YwaNYqsrKzzPZwfHZQAln/+8588+OCD3HHHHRw9epTs7Gw2btzI/v37iY+PZ9KkSaqkX7BJQqfTqfUlwWHBKZVZjh8/7jNSVRAEBL2esJQUamtqmLBuXZdIDebl5bFixQpWr17NrFmzujQK9tNPP/Vo4X/44YccO3aMY8eOsXv3bu666y52797dZWMJ4YeDkIUZQqfw8MMP89Zbb6HX62lubqa2tpbp06fz17/+9XwP7UeDqqoqTp06RWZmZrt1Wkm/3Nxc8vLyMBqNaiTu+PHjgy7p524MjY2NqgVaV1enVmZRAokUfePBgwcHPfdRkiT+7//+jw8++IA333yTgQMHBrV9V/hK91i6dCmXX345c+bMARyyeFlZWect2CiELkHIJRtC1yIrKyvkku1iyLLM2bNnyc3NJTs7my+++AKr1cqoUaO6TNLPHVpaWqiurubs2bOUl5cjiiK9evXqdGUWV1RWVrJs2TJSUlJ47rnn/Jpf7Sz69etHt27dEASBpUuXsmTJEqf1119/PatXr2by5MkATJkyhaeffprRo0d3+dhCOGcIuWRDCOHHDkEQ6NGjBzfddBM33XQT4Czp99prr50TST+j0YjRaKS2tpaLLrqIuLg4dR60qKhIrcyiEGhH3Mi7d+/mvvvuY82aNdx6663nTIhg165dJCUlUVFRwdSpU8nIyODSSy9V17szMkKlwX4aCFmYIYRwgcFisZCXl6e6cQsLCxk0aJBKoIMHD+5UqoosyxQWFnLmzBmGDBnilgwlSXIKJGpubiYqKkolUG+CDpIk8dJLL/Gf//yHv/zlL6Snp3d4rJ3Fo48+SlRUFA888IC6LOSS/Ukg5JIN4aeD6upqFi9ezIEDBxAEgddee40JEyac72GdF3iT9Js4cSIjR470W9KvpaWFgwcPEh0dTVpamt+WqyzLaiBRVVUVDQ0NmEwmNZDIZDJhNBo5e/Ysd955J/379+eZZ57pEqlBb2hoaECSJMxmMw0NDUydOpW1a9dy9dVXq9v897//ZcOGDWzdupXdu3ezYsUK9uzZc07HGUKXI0SYIfx0sGDBAi655BIWL16MxWKhsbGR2CCozFwIUCT9FEUifyX9zp49y9GjRxk4cKBa47Oj0EreFRcXs2TJEiIiIqisrGTRokU88MADmM3mTvXRERQUFKh1LG02G3PnzmXNmjVOij2yLLN8+XK2bdtGREQEr7/+emj+8sJDiDBDCAxff/01p06d+tFp1NbW1jJ8+HAKCgpCc0t+QCvpl5OTw+7du50k/UaPHs0TTzzBpEmTmDFjRtCtPrvdzosvvsgnn3zCrFmzOHr0KJ9//jmCIPDuu++e07qSIYTQilDQTwiBISEhQdXElCRJXS4Iwg+aiAoKCujZsycLFy5k//79jBo1ihdffPGcl3r6sUAQBHr16sXMmTOZOXOmk6Tfv//9b5YtW0ZGRgaiKCIIgkdJv47g9OnT3HnnnWRmZvLxxx87RdfW1dV1iTBBCCF0FF1bDTeEHzWef/55+vbti8ViQRRF9aW9UWqJ9IcCm83G119/zV133cXevXuJjIxk3bp153tYPxoIgkBsbCw2m43du3ezdetW/ve///GLX/yCkydPsmLFCsaPH8/tt9/Oq6++yqFDhzr0O8jNzeXGG2/kzjvv5Nlnn22XimI2m9HrO/5MX1xczM9+9jMyMzMZPHgwL774YrttsrKyiImJYcSIEYwYMYLHH3+8w/2FcOEj5JINwS3q6+vp1asX9fX1/PWvf+Xtt98mMzOTXr16MWfOHFJSUtzuJ0lS0FMYAsWpU6cYP348hYWFAOTk5LBu3Tr++9//ntdx/djw9ddfk5aW5nbuV5H0UyJxjx49SlpamhqJ603Sz2638+yzz/K///2Pt956q8tcroqYwsiRI6mrq2PUqFFs2rTJqfxXKHc4BA8IuWRD8B/ffPONGvzw2WefcfbsWWbPns1TTz3FyZMnef7553nvvfeIjY1l6tSp6n5aspRlGUmSEAThnJJor1696N27N0eOHGHQoEHs2LHjvNZI/LHCmz6qXq9n9OjRjB49mpUrVyJJEkeOHFEl/b755ht69uypRuIqkn7l5eUsXbqU4cOH88knnwRN4MAdEhMT1VQPs9lMZmYmpaWlod9CCB1GiDBDcIvNmzdzzTXXqPNZK1asYMyYMcyYMYNt27YBsG/fPqqqqpg6dSr19fVs2rSJ8PBwbrnlFmw2G3q9vl2+n91ub+fW7Qq89NJL3HbbbVgsFtLS0nj99de7tL+fOkRRJDMzk8zMTJYuXeok6bdp0ybWrFmDJElUVlby8ssvc911153TefDCwkL27t3LuHHj2q37/PPPGT58OElJSaxfv57BARS4DuGnhRBhhuAWH3zwAQ888ADHjx+npqZGLUhbWlqqankKgkDf1tJN77//Pl988QVz586luLiY559/np07d3LZZZexfPly0tLSANwmzNvtdpqamvwukuwPRowY0a7KRAjnDoIgkJqayrx585g3bx6y/P+3d78xNf5vAMffh04UlVgq/RMRy2nGDtaaRVbfkpgSDTHGmDSzPPCE9cTMgwhjYx7EbM2/aSNNf+R/Qk3F5IxCdZIzNX9COef+PejXzVE/+oVOx/d6Par7vs/OddrZfXVf9/W5boX6+nqMRiMzZszo11jev39PQkIC+/btw9XV1WrftGnTeP78OcOHDycvL49FixZhMBj6NT5hP6TpR3Tz+fNntdP0wYMHODg4EBgYCEBjYyNjxowBOk82LS0tKIpCVlYWS5YsQafTkZKSwrx587h//z5ubm5cv34dgB07drBnzx6Kiop48uSJ+n4NDQ1cvHiRp0+f9v+H7Ud79+4lJCSEKVOmkJyczKdPn2wdUr/RaDT4+fn1e7Ls6OggISGB5cuXs3jx4m77XV1d1X/UYmNj6ejowGQy9WuMwn5IwhTdODo6UllZCUBbW5s6mqympobXr1/j6ekJwMSJEyksLOTs2bMEBgYSGRlJYWEhRqOR7du3k5SUREVFhdpQUVRUxN27dykoKGDOnDmcP38egKNHj3Lu3DnMZjNgPavTYrH0OLvT3jQ0NLB//37u3btHdXU1ZrOZnJwcW4f1V1MUhbVr1zJ58mS2bt3a4zFNTU3q96usrAyLxfLLQxnE30tKsqIbjUajNkakpKSo27VaLWFhYfj5+QGdAwIqKyspKChg8+bN6ra5c+dy6NAhSkpKKC8vx8XFhfb2doxGI/v27UOv1zN9+nQuX75MVFQUFRUVVFVVERcXR2ZmJnFxcdTV1eHn56eWcBVFQaPRdD6s+L8/D+S1oD358uULHz9+RKvV0tbWpl6piz/j5s2bnDhxAp1Ox9SpUwHYtWsXL168ADqn9pw5c4bDhw/j4OCAk5MTOTk5dve9Ev1HEqb4oW+XiYwbN45Nmzap+wIDA0lKSiIkJIQ5c+YAMGzYMJqamgCIiIggIiICgPz8fLy9vdHr9bS3t6MoCq2trTg7OzNr1iwSExNJTk7m7du3ZGdnc/z4cUwmEzqdjiNHjuDs7ExHRwdardYuT2g+Pj6kp6fj7++Pk5MTUVFRREVF2Tqsv1p4ePhPqxOpqamkpqb2U0TC3klJVvzQ98tEvj0BeXt7c+LECdLS0tRtS5YswcfHh9DQUGJiYtQZnFevXlU7FBVFobKyEp1Oh8lkorGxERcXF4YMGUJNTQ1ZWVkcOXKE0tJS3NzcuHz5Mh8/fiQlJYW9e/eyc+dOSktLefPmjVVsA1lLSwu5ubnU1tbS2NjIhw8f5CHbQtgZSZii174vg1osFvW+Y5fBgwdz4MABrly5wrp165gwYQLQmTD1ej3QOfLs0aNHhIeHU1NTg4ODA+PHjwc6u23DwsIYP348Tk5O+Pj4UFBQQFtbG/X19Tx+/JjQ0FDOnj1LTEyM+r5msxmj0fin/wR9VlhYSGBgIB4eHmi1WhYvXsytW7dsHZZdyc/PJzg4mKCgoB4nNymKQlpaGkFBQYSGhlJeXm6DKMXfTEqyos/+1zACRVEYNWqUVVdicXGxej/yw4cPtLS0MGHCBJqbmzGZTOrM0OfPn1sNe7979y4LFiygvLycyZMnk5aWRkhICGFhYZjNZqqqqtDpdBw7doyTJ09y7dq1P/iJ+87f35/S0lLa2tpwcnKiqKhInnDxfzCbzWzatImCggJ8fX3R6/XEx8dbDSG4dOkSBoMBg8HAnTt32LhxI3fu3LFh1OJvI1eY4rfrugr9tkw6dOhQdVRaQEAAxcXFeHl5ERAQgMViYfny5dy+fZsVK1ZQUVFBc3MzN27coLa2lvnz51NWVkZwcDD+/v4oioK3tzfPnj3j3bt3lJaWUllZybZt27q970Axc+ZMEhMTmTZtGjqdDovFwvr1620dlt0oKysjKCiIcePG4ejoyLJly8jNzbU6Jjc3l5SUFDQaDbNmzaK1tXVAVx2E/ZErTPHH/Kg5p2ufm5uburyio6OD9vZ28vPz+eeff/Dy8iIzMxMPDw8ePnxIYmKi1TMSNRoNrq6ubN68mdTUVPUhvwO1KSgjI4OMjAxbh2GXGhoa1O5sAF9f325Xjz0d09DQoI7HE+JXScIUNtU1b3bw4MFotVq0Wi1Hjx4FOhtl3N3defnyJR4eHowePVp9DUB0dDQbNmzAxcWFhIQEm30Ge7NmzRouXLjA6NGjqa6uBuDNmzcsXbqUuro6xo4dy6lTp3B3d7dxpF/1VDX4/h+j3hwjxK+QkqywKY1GYzUuT1EUtZHI3d0di8WCn58fWVlZ6ni+ruYjR0dHdRB812vFz61evVqdB9xl9+7dREZGYjAYiIyMHHCPQ/P19eXly5fq7/X19d3WsfbmGCF+hSRMMaB8n0AHDRqkJsKuJqOmpiaqq6u5du0aGzduZNKkSeprxc/Nnj2bkSNHWm3Lzc1l1apVAKxatUqdwjRQ6PV6DAYDtbW1tLe3k5OTQ3x8vNUx8fHxHD9+HEVR1CVJUo4Vv5OUZMWA930ivHfvHunp6WzZsoWVK1cCXycBib559eqVmly8vb1pbm62cUTWHBwcOHjwINHR0ZjNZtasWUNISIi6znfDhg3ExsaSl5dHUFAQzs7O8oQa8dvJA6SF3ZIk2Xd1dXXExcWp9zBHjBhBa2urut/d3Z2WlhZbhSeErfV4YpGSrLBbkix/H09PT3UJhtFoVBushBBfScIUQhAfH092djYA2dnZLFy40MYRCTHwSElWiH+Z5ORkSkpKMJlMeHp6kpGRwaJFi0hKSuLFixf4+/tz+vTpbo1BQvyL9Fi+koQphBBCWJN7mEIIIURf/WxZiXRVCCGEEMgVphBCCNErkjCFEEKIXpCEKYQQQvSCJEwhhBCiFyRhCiGEEL0gCVMIIYTohf8Ae0grjPe7ZuwAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/markov_prop_7_0.png" + }, + "scrapbook": { + "mime_prefix": "application/papermill.record/", + "name": "flow_fig" + } + }, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAAFUCAYAAACp7gyoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9eXAc133v++1lVswAxMoFoLiTICmRoiTKpBL7pm49W0tu+T7p1q34+darih0+SbEr0XXi1FPixIkc2+9ZdhRH0Y3kWJRtyZIVP8eWnNhSJMuSvMgiRVHURoLY930AYnrWnl7eH4PTON3TPdOzAGgA51OFAjDT3XNmpvt8+/c7v4XTdR0MBoPBYDCKw6/2ABgMBoPBWAswwWQwGAwGwwVMMBkMBoPBcAETTAaDwWAwXMAEk8FgMBgMFzDBZDAYDAbDBWKJ51nOCYPBYDA2Gpzdg8zCZDAYDAbDBUwwGQwGg8FwARNMBoPBYDBcwASTwWAwGAwXMMFkMBgMBsMFTDAZDAaDwXABE0wGg8FgMFzABJPBYDAYDBcwwWQwGAwGwwVMMBkMBoPBcAETTAaDwWAwXMAEk8FgMBgMFzDBZDAYDAbDBUwwGQwGg8FwARNMBoPBYDBcwASTwWAwGAwXMMFkMBgMBsMFTDAZDAaDwXABE0wGg8FgMFwgrvYAGIzVQNd1aJoGRVEgCAJ4ngfPs/tHBoPhDBNMxoZC13Xouo5cLgdVVZHJZAyh5DgOgiBAFEWTiHIct8qjZjAYXoDTdb3Y80WfZDDWEpqmIZfLQdM0cBwHXdchyzJ4nge5Doig0giCYAgpz/MQBIGJKIOxvrG9wJlgMtY9xPWqqiqAvCXJcRw0TTME0wkioNbrhAgn/UOOy2Aw1jy2FzJzyTLWLbquI5VKmVyu5Qqa3T5EPBVFQVdXF1pbW1FfXw8AJpcucesyEWUw1gdMMBnrDl3XoSgKFEXB2bNnceLEiZoG9BABpH8LgmAEEmWzWdP2xBql10aZNcpgrD2YYDLWDbquQ1VVKIoCXdcNUVopYSpmjeZyOciyXCCyLMCIwVg7MMFkrHnshJJ2w5LHlgNy/GLPA3lXrdOYaXieL3DpsnQXBsMbMMFkrFmICzSXyxUIJYEE99iJTjKZxPz8PCKRCMLh8IoKk5M1SiJ3rdsya5TBWH2YYDLWHHQuJUkRcRI7Owswk8mgt7cXiUQCjY2NiMViSKVS4DgOdXV1iEajiEQiiEQiEMXil0gpC7Mcirl07axRlu7CYKwsTDAZawprLmWpNUo6xzKXy6G/vx+xWAx79uzBwYMHoSiKsb+qqkgmk5AkCVNTU+jr64OqqgiFQoaARiIRBINB01pkrQTTDmuAEYFY16qqGhZpLpdDIpFAW1sbS3dhMJYBJpiMNYFTLmUpOI6DoigYGxvD+Pg4duzYgX379oHneWiaZlrfFAQB9fX1RooIkBemdDqNRCIBSZIwMTGBTCYDURQRiUSQTqfh9/vR2Ni46i7dbDaLiYkJNDU1mQKMyHtj6S4MRnWwwgUMT0NSRC5fvoyrrroKfr/f9USv6zpee+01aJqG9vZ27NixwxR8Q6zVSoSDWHODg4MAYLhLw+GwyaXr9/vLPnalpNNp9Pb24pprrjE9Xqr4Akt3YTAKYIULGGsHOpcSAOLxuOtoV13XMTMzg97eXqiqiqNHj6KhoaFgu2qEwefzobGxEfPz86ivr0dLSws0TUMqlYIkSYjFYhgaGkIul0MgEDAENBqNIhQKragosXQXBqM2MMFkeAqnXEp6LbIY8/Pz6OnpQSgUwrFjx3D58uWSgTvVQK9h8jxvCCP9frLZLBKJBBKJBKanp5FOpyEIQkGAkTX1ZDmpRboLs0YZGw0mmAxPUCyXElhKD3EikUigu7sbAHDo0CFDtJY7KKcUHMchGAwiGAyipaXFeFxRFCPAaGJiAolEApqmIRwOm6zRcl3QtRgvS3dhMOxhgslYVdzkUgLOwpfJZNDT04NUKoX9+/ejsbHR9Lxby7RSKhVkURTR0NBgchWT2reJRAILCwsYGxtDNpuFz+cziWixnNHlECuW7sJg5GGCyVgVysmlBGBEtRLoFJG9e/eitbXVdlJebQuzHEgeaF1dHTZv3mw8Lsuy4dIdGhpCKpUCAFOqC+0GXqmx0r8JdLpLPB7H+Pg4du3aVdDZhaW7MNYiTDAZK065uZTAkvCpqoqhoSFMTExgx44d2L9/f9F9VyJPcrkF2e/3o6mpCU1NTcZjqqoaAUYzMzMYGBiALMtQFAX9/f2GNUrnjK4E1u9SlmVjnVRRFJbuwljTMMFkrBiapkGSJCPVohwLg+M4TE5OYnZ2Ftu2bcOJEydcBcmUWvusltWyYAVBQDQaRTQaNR5LJpPo7e1FNBqFJEmYnJxEJpMxtiWWaF1d3YoEGJHPhbZGaS9Cqe4utFuXWaMML8AEk7Hs0Cki58+fx8mTJ8sKZJmensbU1BSam5tx4403wufzuX7tteSSrRYShNPa2orW1lbjcZIzmkgkMDY2hmQyCV3XbQOMlmNMxZ5j6S6MtQQTTMayYc2ldFr3cmJubg49PT3Gmt7mzZvLEkvyWmvdJesWp3GQnFE6IIrkjCYSCczPz2N4eBi5XA5+v98QUFKUvlJRqqRLjJt0F/q4LN2FsZIwwWTUHDuhpCewUhOpJEno7u4Gz/M4fPgwIpEIuru7KxKm5Y6SBWqTzlEr3AqFU84oCTAia6OpVMq0rdui9OR4tRIulu7C8AJMMBk1w6noAE2x/pSktFs6ncb+/fuxadMm4zlrlKxbVqo4+nqA4zgEAgEEAgE0Nzcbj6uqarh07YrSE2s0EAiUdWNUi/GWm+4C5IOo6EhdBsMtTDAZVVOq6AANET76eVmW0d/fj/n5eezduxctLS2OQlsuG8klu1wIgmCbM0qK0tM5o6IoGgK6Gp9LqXSXS5cuoaOjw7Cs6QAjlu7CKAUTTEbFlJtLCZgtRVVVMTg4iMnJSezcuRMHDhxwnKiqsTCXM0p2o8JxHMLhMMLhMNra2ozHc7kcJElCIpHA3NwcJElCPB5HXV2dKcCo3LXoWoyXnFvEVUsEnaW7MNzCBJNREZXkUgJ54VMUBVNTUxgeHkZ7eztOnjxZUmi9bGEyQV7C5/MZOaORSARzc3PYvXs3kskkEokEYrEYBgcHoSiKUZSeWKQrUZSe9m6wdBdGuTDBZJQFyaWUJAnNzc1lTRgkQOP8+fPYvHlzWSkiXl3DZDhDu+etOaOkKD2xRqempoyi9NYAo1rmjFqXA+woN92F53n4fD4WYLQBYILJcAUd+ZpMJjEzM2PK9StFLBZDb28vZFnGkSNHCmq+lqJS4fNqLdnlwCvjIBQL+qGL0tPnkaIoRoCRXVF6Yo2WU5Te7ZiKUSzdRdM0ZDIZlu6yAWCCySiKXYqIKIqurb14PI6enh4IgoCrr74afX19FbXb4nkeuVyu7P28JGgrgZcm5ErESRRFbNq0yRQhrWka0uk0JEnC/Pw8RkZGIMsyfD6fqYJRsaL09LFKbVMOLN1lY8EEk2FLsVxKN+7RVCqF3t5eZLNZ7N+/34iwXGnXaqk1xmonq40myOVQq8+F53mjKD0N3Wc0FoshlUoZBexpa5S+Qau1YNpRbrqLLMsIBAIIhUKsu4vHYYLJMOEml7KY6MmyjL6+Ply5csU2RaRSwfRylCwTTGeWc+J3yhklAUZ0zmgwGEQ0GoUsy8hkMisSYERTLN1lZGQEzc3NpvOIeHJYuou3YILJAFBZLiWNoigYGhrC5OQkdu3ahc7OTtuLu9I1RS9HyTLsIRHUK4kgCKivr0d9fb3xmK7ryGQykCQJmqaht7cXmUwGoigW9BldiaL0NOT8JEFDZLxA/pqyLkPQUbos3WXlYYK5wSFBC4qiuM6lFATBEExN0zA6OoqRkRF0dHSUTBGp1OJbLldutdVomEu2OF6YzDmOQygUQigUwtDQEI4cOQLAXJR+ZGQEyWQSABAOh01ro8tRlJ5GVVXTNVOq+IKqqqb1UVZ8YeVggrmBqTaXcmJiAgMDA2htbcUHPvABV8E8K72GuRK1ZL2C197ncpfGq5ZiReklSUIsFsPQ0BByuZyRM0oHGNXqvWma5rpVHUt3WV2YYG5AiEV58eJFo7pOORfS/Py8EbF4/fXXIxAIuN53NdYwN0paCeANi47gdcG0w6koPR1gZFeUPhqNoq6urqIIcKuFWQ4s3WVlYYK5gbBGvs7Pz5d1ocbjcXR3d0MURYRCIRw6dKjsMXgpSpZMEtVM7F4TTC+xFgXTDjpntKWlxXic5CRbc0ZDoZDJpWstSm/FrYVZ7phZukvtYYK5ASjVbqsUqVQKPT09yOVy2LdvHxoaGvDaa69VNBavWZhM8JYPXdeXPYVjNRFF0bYoPekzShel9/l8BQFG5LOpxsIsh3LTXeLxOFpaWowygORnI8MEcx1TSiiJ9eV0EWSzWfT19SEejxspItWy0tGuTvuRwu9jY2MQBAF1dXWGVRCNRl271pjgOrNeLMxyIHmgpOk5gfQZTSQSGBoaQiqVAgDU1dUhm83iypUrq1aUnv5N0HUdPT09qK+vN1mk1nSXjWaNMsFch7jJpQTsW20BeVfT4OAgpqamsHv3bhw8eLBmFwTP81BVtaL9amFh6rqO8fFxDA4Oor29HTfeeKNhFUiShOnpafT390NVVVM5tmg0uuzRkusNrwnmat7Y+P1+oyg9QVVVpFIpXLlyBbOzs0ZR+mAwaAowWumcUWDp5pp2FbtNd1nPxReYYK4jysmlBPInuaqqhjWlaRpGRkYwOjqK7du3u+oiUi4rXeKOjpKNxWLo7u5GY2OjUfhdlmXoum5bHJy41ubn5zE8PGxES9KWKNnWC3hlHAQvCqaXxiMIgmFV7t+/H8BSziixRicnJ5HJZIyi9OTcq6urW9acUbsc2krTXc6dO4ejR4+ayh2uVZhgrgMqyaUElvIpdV3H5OQk+vv7sXnzZtcpIpVQzRpmpS5ZWZbx5ptvQhAEHD16FOFw2NV+VteatcPG5OQkEomEEUxBJjM3NU2XCy8JgtcEaiXK4lULnTNqV5RekiSMjY0hmUxC07SCPqO18oKoqupakEuluzzwwAO4//77mWAyVp9KcymBvAjNzs5idHQUmzZtwvHjx8u64CqZEFeyxF02m8XAwADm5+dx7Nixsjuk2I3B2mFjfn4eU1NTaG5uNnL3SE1T2p273BaBF2GCWTucitLbeUH8fr9JREmN2nIoRzDtoNNdEomEKTBqLcMEc41ChPL999/HgQMHyl4zWFhYwNzcHHK5nGuri4a4SCsRzEpdq24FkwT0TE5OYtu2bRAEoWqxLIYgCLbrU1aLQNf1gsLgKx3ksZIwF3FpqvmMnHJGSYCRJEmYnZ01ckbpwDZrUXoriqLUzMskSRITTMbqYI18JROx24kgmUyip6cHiqKgsbERO3bsKFssAeeAoVJUWhrPzRqmNaDn5MmTSCaTkCSp7NerdlyCIBSkHGiaZuTtzczMGMFFdN5eNBotqxCE1/GSQHnRwlyOdmNORenJuihdlD4UCpmsUZIzWq2FSZPNZhEMBmtyrNWGCeYawSlFhATulLJU6BSRffv2obm5GV1dXRV38ljpfMpSlqldQA+w/HmY5ezP87zhot26dSuA/PdKej0uLCxgdHQUsizD7/cb265WpGS1eM2i2wiC6YTdDRw59xKJBOLxOMbHx5HNZiGKotHzVpIk1NXVVTxGr3kZqoUJpscplUtJBNMJRVEwMDCAmZmZghSRUvsWwyttuhKJBC5fvuwY0OP1WrIcxyEcDiMcDpuCi2RZhiRJkCQJU1NTSKfTRncNIqR2wUVeEiivCabXxgNUv1ZYDfS519bWZjyey+UwOjqKhYUFU1F6OsCo3KL0XvvcK4UJpkdxm0vpJHqapmF4eBhjY2PYvn07Tpw4UTC5Vipe5HVXs2JPNptFb28vEokEDhw44BiBtxZrydJuNbpYBOmuIUmSkfxOgosikUjF7u7lwmsCtZEtzHLw+XwIBoMQRRHbt28HYF5OiMViGBwcRC6XM3JGnTwhXnx/1cAE02NUmktJ70+6iGzZsqVoiki1FuZKFyAAzAE9e/bswaFDh4pOyl4TkWqw665Br03FYjEsLCzg7NmzBZWLViO4iAlmaVbTwiyGdVz0cgKBLkpPe0JIzuirr76K7du3m/YpxvPPP4977rkHqqri1KlTuPfee03PP/nkk/jKV74CAIhEInj44Ydx9OhRAMDOnTsRjUaN4gnnzp2r9iOwhQmmR6gml1JVVei6jtnZWfT29rpOEanGwlzpIurETfn6668bAT1uPp+1aGGWA702FYlEMDk5iX379plaVNEVZMikR3L2llPQvCiYXhoP4E0RB+AqLqJYUXoioo8//ji6urpw7NgxHDhwAEePHsXv/d7vYffu3QWv9+lPfxovvvgiOjo6cPz4cXz0ox81NXjYtWsXXn31VTQ2NuK5557DnXfeiTNnzhjPv/zyyzUp31kMJpgeoNpcSkmSMDAwgEAggGuvvRahUMjVvoIgVFR1h7zuSrlkSUCPpmmmgB43rHfBtMMp3SCTyRjBRaQoOMnZIyJay+AirwmmF4vBrxULsxxIzuif/umf4tKlS3jggQfw5JNPoqenBxcuXLD1TJ09exZ79+41hPRjH/sYnn32WZNg3nTTTcbfJ06cwOjoaEXjqwYmmKsIEcpLly5V1JcymUxiamoKHMfhyJEjrl0fhNWwMMvZzxrQc+HChbJdi14P+lkp6AoydIAH7VKbnp42XGq0O7fSKEmvCaYXrTkvjgmoXR5mPB43XKWdnZ3o7Oy03Y7EWhA6OjpM1qOV06dP49ZbbzX+5zgOH/nIR8BxHO666y7ceeedVY/dDiaYqwBxvZI7rbm5ubIumkwmg76+PiQSCTQ3NxesLbjFq1GybgN63LDRLMxyBcouZ09RFKP8Hx0lSVuikUikpAXiNcH02niA9Wlh0sTjcdTX15fczu4acvquXn75ZZw+fRq/+tWvjMd+/etfY9u2bZiensaHP/xhdHZ24kMf+lDlA3eACeYK4pQi4vYizuVyGBgYwOzsrBHwMjY2tuKiV82+xfYjAT2kS0qpgB43rKegn1LUSrhFUbQNLiJFIEgNXdLRhV4XpT0AXhMoL1pzXhwTUDvBdFvlp6OjAyMjI8b/o6Oj2LZtW8F277zzDk6dOoXnnnvOdJNHtm1ra8Ptt9+Os2fPMsFcq7hp4FxsclFVFcPDwxgfH8dVV11lShERBKGgg7pbqrEwK93XziKzVuixS4GpFDeFC5bz+OsFQRBQX19vshboWqbW4KJIJIJMJoNsNotwOOwJ4fSiOK13C1OSJFfer+PHj6OnpwcDAwNob2/H008/jaeeesq0zfDwMO644w488cQTRncXAEYh+mg0imQyiRdeeAGf//znqx67HUwwl5Fycymtawa0kGzduhUnTpwoOIlXw61azb7W9+9UoadWeGGiXq/QwUVbtmwBYG5PNT09jcHBQfT29sLn85nWRVdDRL0aJbtcnYGqoZZrmOTcKIYoinjooYdw8803Q1VVfPKTn8Thw4fxyCOPAADuvvtufOELX0AsFsOnPvUpY59z585hamoKt99+uzHuj3/847jllluqHrvtOJflqBuccnMpRVE0naC6rmNmZgZ9fX1oamoqmiJSaQGBavetRmyB0hV61gobxcJ0Cx1cNDk5ib179yIUCpkqF5GC4HSPx2qCi9yi67rnrDmvWpi1ssYTiYTr+IrbbrsNt912m+mxu+++2/j70UcfxaOPPlqw3+7du/H2229XN1CXMMGsISSXMpfLuRJKAm0lXrlyBd3d3QiFQq5SRCotIFCLfStJSclms0in00aXlXIDery2LsZwhv6u/H4/mpubC4KLSISutQQbHVxUKwvMiy5ZL46JUIvrbD11KgGYYNYE0kCYzqUs5yIQBAGSJKG7uxu6ruPgwYOu78qqXYdcKQuTDugRRRE33nhjxb00a31HXo0Ie8nC9Mo4CKU+V6cej3ZdNcLhsMkaraRRshfFyYtjqiWSJLmKkl0rMMGsEk3TMD8/D1mW0djYWHYuZSaTwZUrVxCPx3Ho0KGy+zauRnk7sq8bwbQL6Hn99dcrEigviRPBa2PykvVdab9Ua3CRrutG5SK6UXIgEDBZosFgsOjredE74VWXbK1wm1ayVmCCWSF0LiVxK9ENhEuRy+XQ39+PWCyGuro6dHR0VNTkeDUiXQF3glnrgJ5q100ZK0utBIrjONTV1aGurs507Gw2a6yLTkxMIJPJQBRFU5oLHVzkRWvOi2Oq5Q2gJElV5VF7DSaYZWKXIuLz+VwLD50ismPHDuzfvx/9/f2rErhTjXVUzDpdroAer1lzgDfH5BWW06Kj65i2trYaj8uybNzAkuAiUjg8mUwiFAp5yqrz0lgItRwTW8PcoBTLpSRRrqX2Hxsbw9DQELZt22ZKEVktK7GaycxOqMup0FOpu45ZmGuH1XCB+v1+NDU1mbw9JLior68Ps7OzmJiYgK7rBR1dViO9w4sWZi0FM5PJIBgM1uRYXoAJZgnc5FIWEy2SItLb24vm5mZb12S1orcaIkKLV7kVekh913In0+XqPVnt/szCtMcra4YkuCgcDqOjowPRaNTo7yhJEmZmZtDf3w9VVREKhUzrooFAYFnH5kULs1Y5mCQY0ms3BNXABNOBcnIpnSzM+fl5dHd3o66uDtddd53jnZYgCMhmsxWNc7UmJOKSHRsbK7tCDxHbci8kL1qYXhJMr4yD4LXx0OecU3/HdDptCi6SZdnUJDkajZYMLqp0TF6h1iLuhZumWsEE00IluZRWC1GSJPT09IDjOBw+fNjUZskOURSNHLS1wsLCAmKxGILBYEUtt1ayl+ZGwmuTk5fGU8ri5TgO4XAY4XAYmzdvNvahO7pMTk6agovoykWVdnRZr4K5Hq9VJpiLVJNLSSzMdDqN3t5epNNp7Nu3z3XUazUu2ZWGBPQAQDQadWzXU4yV7KVZCk3TqrrLZyJeHC8JZiXfs1OT5FwuZ3R0GRoaQiqVAsdxBZWLvOZudUOtBDOVSpkim9cDTDCx1InB5/OV3UEEyF88mUwGFy5cwJ49e9Da2lrW/qspmMTaKzWRWAN6wuFwxeWoqqlDW6k4Wa0LXdcxMTGB/v5+4/Fy21cx1ha1dH/6fL6C4CI6xWxsbAzJZBK6rps6ukQikZrXS641te6FuZ7Y0IJJcikVRcGbb76JkydPliV0qqpiaGgIExMT4HkeJ06cqOiOuhaCWelkUGo90SmgR1VVT7UGKwYRWvLdLCwsoKurC9FoFDfccAOAvICSIJCJiQkkEglTJGV9fb2tiDILc+2w3EFIgiCgoaHBlEZBOrqQ4KKBgQEoioJQKIRIJAJFUZDNZuH3+z1jjdeyU8l6KloAbFDBtKaIELFwe8JqmoaxsTEMDw8bwS5nzpyp+ISvVjBJikclgummU4pdQE+1VYIqEZlKxYm8XiaTQXd3N2RZNtaWVVVFLpdzbF+VTCYRj8cxOTkJSZJMIkrWrhhrg9UIsKE7umzduhXAUnAR6Sl66dIlyLIMv99vskRDodCqiKiqqhWVHrTitrXXWmJDCaabvpSl9p+enkZfXx9aWloKgl0qvYOthWDaiZ7bfa1Wm5sKPdVcyKuxhkly8Pbt24eWlhZX47eLpKTTEaampox1rPfff99UYWY1cvqYpVscr0SkkuCiUCiEoaEhXHvttdB13dTRZWpqCul0GqIoGksFkUhk2Tu6ALWzMNdbWTxggwgmSREh3TXKFUoAmJubQ09PDyKRiG2KSDWi5abwQTFqVU92pVpurVSUrK7rmJycRDweR0tLS00aU1tFVNd1vPHGG9ixY0dBTl84HDZcudFodEXWrrzi1vMqXvp8aAHnOA6BQACBQKAguIisiw4PDxvBRdaOLrVcb1cUhblkHVj3gknCwos1cAacg19IFxGe54umiFQjmNVGf1YrmJlMBkNDQ64q9NSClbAwFxYWcPnyZUQiETQ0NKCjo8NWLGtRuIBER1rdbqlUCvF4HLOzsxgcHISiKKYAkJUSUYY3cWPJ+Xw+NDY2miLuSZAivd6uaVpB5aJKz61K5zEr660sHrABBNNt1KsoilBV1ZhU0+k0enp6kM1msW/fvpIiQqzESiqDVDtpV1pPlkT1Xbx4Efv37y9ZoadWLGeUbCaTQU9PDzKZjNEm7c0331xxdyVdMNwqopIkIRaLGSJKV5eptHUVY+1RTdyB3Xq73bkVDAYLKheVusZr6ZKl+5+uB9a9YALuJloieLquo6+vD1euXMHevXtdr3etZmpIuQE4dECPz+fD3r17TQWsl5tKg36KCS0dzUveD/neKn29WkOL6JYtWwDYV5fJ5XLGRFdfX89EdJ1Sy4o6dHARgQS6SZKEhYUFjI2NIZvNwufzmUSU7uhSy3FJkoRdu3ZVfRwvwQST2mZgYADz8/PYtWsXOjs7y7K2ql2HrIZyxNoa0DM4OLjiYlJLC1PXdaPRMClqb71r93Lqh1N1GTLR2Yko+VnuOqeM5WW5g5A4jkMoFEIoFEJbW5vxOF25aHp6Gul0GoIgGAKayWRqMi4W9LMO0TQNo6OjiMVi2LZtG06ePFlVekY1LGeUrVNAT7Xrp5UWUa/FGmY8HkdXVxfq6upwww03OArIahWorxS7ic5qLYyOjkKWZVMTZeIhYawNNE1bleIYJLiIdpcqimJEfGcyGaMoCR2hG4lEylrbZEE/axS7CZ1YJv39/WhtbUV7ezuam5srvrOq1sKstIMHUFwwS7XcqkUT6XIv+mosTE3TkM1m0dPTg3Q6baxTltpvrQuJk4jSTZRjsRgymQySySQikYjhznWzbsVYeeiYidVGFEUjuGhychLHjx83BRcRLw6J/qZduk7LBUww1wmxWAw9PT2or6/H9ddfj0AggP7+/lVL7SD7K4qydPKpCrhLvwB/+RdAKgZtxxHoJ/9PwFfY8cTutd223KrGwlxpwQSAmZkZjI2NYc+ePWhra3MlBOtBMO2wNlGur6/HwsICOn0VvFsAACAASURBVDo6IEkS4vE4xsfHjUoytDu3lh031gJe/P69khfqhF1wkTVwbWhoyFguoPuKNjQ0uI6Sff7553HPPfdAVVWcOnUK9957r+n5J598El/5ylcA5C3ehx9+GEePHnW1b63ZUIIZj8fR3d0NURRxzTXXmAoDV2shiqJo5HlWgkn0pocgfucvwF94HlCSAAcIAoBv/hEgAnqoDtqOa6B96A+gf+DjpvZgdKNqNy23BEGoeNzVpIeU85p0wYi6urqy8ynXq2Da4ZTPR1uipOMGHfwRjUZXrbLMSuCV3pw0XuyFWQqnwLVMJmOsi37nO9/Bj3/8Y+RyOXz1q1/FiRMncOzYMezfv7/g/aqqik9/+tN48cUX0dHRgePHj+OjH/0oDh06ZGyza9cuvPrqq2hsbMRzzz2HO++8E2fOnHG1b63ZEIKZTqeN8lP79++3vesRRbHinpRk/3Q6XfH+hmDOjEL8u1Pge98E5BTAIf/DL/3m0kkI0usQel4Hnr4bbYf+N4z+9l8hFqsvWaHHSi0szEr2cytgkiShq6sLoVAIe/bsqSggodTreW0iXQ7sRJSuLEOCP0RRNIot1NfXVyyiXrtB8aJgetHCrLRkJVkuaG1txX333Yf77rsPN910E2699Va88847+PGPf4zOzk78zd/8jWnfs2fPYu/evdi9ezcA4GMf+xieffZZk+jddNNNxt8nTpzA6Oio631rzYYQzEwmg/b2dtNkYaXanpQ1KW+XSkB4+u/Av0eJJbmeBJj/J1qoqQj3/Ad2TJ/D4I2fw9EP/j60eByZmRn4tm1b1nFXWk/WjdDKsozu7m6kUil0dnaivr7emNTLpRILc24ujcbG9e229Pv9aG5uNgV/0CI6MzNjiChtiVrTEOzwmkB5UZy8aGHW8nPieR633HILbr31VsdtxsbGsH37duP/jo4OnDlzxnH706dPG8crd99asCEEs6mpqaS7tRYu2WrXQIULvwT/8o+AdAYg8ztH/eYB0EuY+tJPIBvDgbf+Grn2BlzqCuD1Bx5AoKEBzZ2daDl4EC2dnWju7EQdFV4OVGdhCoJQ8R2p02tqmoahoSGMj49jz5492Lx5symfsppgITdomo4f/KAbX/vaWSiKgs7OZlx//RZce20b9u9vQkdH1FNCUGvsRJT0fpQkCbOzs0ilUkYaAi2i9ETLBLM0mqatSs3hYqx082i77ZzOm5dffhmnT5/Gr371q7L3rRXe+raWCTcfIqn0UynVWpiB5AICrz4D7socoFCTO3HJAktWJbAkqPRvWYLvpc9A9H8CifFxSKOjuNLXh6Gf/xxCMAheEBBqbsYd3/8+fKFQ1eOupYWp6zpmZmbQ29uLzZs348SJEzVrpVVqPzK5x2JZ3Hffefzrv/ZAkjIAVPT3j+G550YQCPAIBnls3RrC97//v2P7du9F/y2XG9Su9yNd43RoaAjJZNIkoqHF88sreE3AAW9amLWqI0vOxVKfeUdHB0ZGRoz/R0dHsc3GM/bOO+/g1KlTeO6554ybObf71pINIZhuWG0LMzQzhuCFXwHZXF78aMsSWHLFFkMHkI0jMPgtaDIHOZlBNh4Ht3hnzft8kMbG0PfTn6Lzv/23/GOrtIZJ70fWKYPBoG1h+2pfr5hgkrKJ77wTxx/90Zt4990YFMWP/KWhLb6eglRKQyqVA89nkMmsToEKN6yUKNjVOCW5fMQSTSQSOHfunMkSXYluG3Z41cL02phqVUc2m826umk6fvw4enp6MDAwgPb2djz99NN46qmnTNsMDw/jjjvuwBNPPIH9+/eXtW+t2RCC6dbCXM20kpZnHwMfmwcU3bxWSQunnVVJ/15E1OPgcjxUWQc4Ln8InoeWy0HNZND1/e8bgllq3GOvvw4hEEDzgQPwWbqXVBv0I8syent7IUkSOjs7S4agL4eFmctp+O53J/D5z/dibi4HXQ8D0BZ/VOonB0ABoEPTvBXQ4hXoXD5ZlnHx4kVcffXVhiU6MjKCZDJpFKunc/mWWzi8Kk5eszBrWUfWTS9MURTx0EMP4eabb4aqqvjkJz+Jw4cP45FHHgEA3H333fjCF76AWCyGT33qU8Y+586dc9x3OdkQgumG1bQwuXffgDgzAU5R8/M0kJ+jgbxQkvO32DxNntMAUQQ4bvFAmg6d4wBVzW/CcRh55RX0/PCH2HfHHSVFr++559D/4ovgeB71V12F5gMH0HroEJo7O6FXYZ0uLCzgjTfewO7du3Hw4EHX/SmraSBtJR5X8Zd/OYQnn5xAMikCCCH/wetYEk0FQBp5wQR0HUwwXUBcoKIoYtOmTaaCGaTovyRJGBsbQyKRMNIVSLGFurq6moqJF12y61nEyylacNttt+G2224zPXb33Xcbfz/66KN49NFHXe+7nDDBXKTaEnFVFfiOTcM3MwmoWmGwjw6zO9Y6RJuXzAsmvY1u+juXSmHgpz/F7t/93ZKdTvjFQgq6pmFhcBALg4Po/4//yA8xGsXN3/mOize4xMzMDLq6usDzvO06ZTFq2Ufz0qUs/uRPxvHrX0uQ5QAAP5ZEkghlDnYfMBNMdzgJlCAIaGhoMHkUSFWZeDxuiCiAAku00sl8PYtTLWHNo4uzIQTTa3eWVsR/+hKQSsA0OdNzsnV+LuWSFSyCaUFTFIy98goy8/PwNzcXdcmKRQp8a7LsWsASiQS6urrg9/tx+PBhDA8Pl31hViOY9H7PP5/GvffG0NOjQVVpF6yCJVesDLOZnydvYZY9hA1Hud+TU8sqYomSvo+6rpuaJ0ejUVfnkRcF04tjUhSlJmuYbl2ya40NIZiAh6u9jA1D53hwmlbc5UpTYjvRlxdMzmFTXdeRnprC+b/9W/zWP/5jUcHk/X6kZmYg+P0Qg0EIfr+hxlouV3Ldll6nJLVs0+n0shc8oKG/+0ceyeIrX0ljctIHXedR6ILVQLtg7fCqhem187vaG1We521FlK5v2tvbC13XEQ6HDXeuXZFwr7pkvWhh1qp5NLMwNwDVXljl7s+f+QXQ2wVOUZwjYUm+pdNzlt9uznddVTH3zjvIxeNFJ1oxEEB2YQHa4vosx3EQgkGIwSC4QACp6WmASh4maJqGkZERjI6OYteuXaZ1ylpF17qF4zioqo7PfU7DN7+pIx4PIJ+jo2FJLBXkxTMN8xdR+F16VTAB73hTlkugeJ43LEuCVURJkXDaEs3lcp6z5rxUfJ2gqmpN2sYxl+wGgESMVnqHRSwZtxOFtLCA5Lk3sE13lzUCwBTcY+u21W3WMO0Oo+tYuHQJwz/6EXDggON2gt9vej+6rkNJp6Gk09A1DT3f/S4OXH+9aZ+ZmRn09PSgra3Ndp2yFpZiOSSTPL74xVY895wf6bSI/IdFol/pv+lKT3YfYP71VZX5ZEuxkhadk4iSIuEzMzOYm5uDoihIpVImd66b8pHLhVddsisd9LOW2DCC6WayJZGulQom2d+p3Q2BtKdKJhK46ZcvQMiklyJhnXAZIQsAAhFMJ58sAE3XoWQymHzpJXBXXeV4aMHvB5wuao6DkskY/5Kem6Io4tixY455WNX0wyxXMEdHgc98Ziteey2EXI7O1yHfMbEuU2R09EgLjqfr5hgqhj2r7QLled7o4bh161ZMTk4im82ipaXF6LQxODgIRVEQDodN7dBWSkR1XfecYNbSJdvR0VGDEXmLDSOYbqhVaomTYFpbbh1u3wY+GLbd1oRFEEuPo7SFCeTdsrO/+Q2a/ut/ddxGCAQATYOazYLj+fwkSH4DUDMZ5HI59Pb2YmFhAQcOHDAls9uxEiXuAOD9LuAPP+3D+Td9UDTOHHls/CPA9QcLllbiltUWTCtEnOw6bdDtqoiIhkIhkyVa6iZ4vbAaaSVrCSaYFMtVvMCp5ZZw+n+BGxkEdLW0hWkczPLbIc1EcLOOqevIzs4i9cwzwH//77bbCIEAsGjZ6dR74wCA4zB74QJ++W//hgMf+AA6OzuXtT9lOUJ75k0Of/hZEZcvc1BF5A1LEgBL1oQrrDNhN3avCcRq47XPw8n96dSuKp1OQ5IkzM/PY3h42Oj5SAS0vr5+XYooE8zibBjBXIlqP3b7x2IxdHd3o6mpqaDllh4K5U1BlXPv57MNe7UbS1GP7NKuqgqlu9vxeesaprHfom8yPTqKq/x+tLe3l3ilJSqdSN0K7Su/4XDX/y1idJQSS2sgbL5UrPUVXI1DVZmFWQqvCWY57k+O4xAOhxEOh7F582Zj/0wmA0mSsLCwgJGRkQIRjUajNQmYWU3YGmZxNoxguqGWFqYkSeju7oYgCDh69CjClrJySCbA/fiHgJIvt1Y19CG0fGqJ0/xPG6mqokDp6cHIk09i+//4HwXbCn5/vhZtMaGqoiRgOZSagDUN+NnrHE59TsTsPAdtqSSsOXtEhlttpF8dAFnDNH8W5QZ7LRdeSivxwudBU21nELrnY9tixx+riI6OjkKWZQQCgQIR9dJnUYxarmHS1Z3WC0wwKWphYabTabz33ntIJpNG3qEtPj/0wT5Als1dSNxAImSLzI+lznlTgK2qIjUwYLudIZiAWTTJBKBp0GS51IiXHVUFnnmVwz1fERFLctB9WBJL8nmRrJEq0DQN8/MLkOXNnnTJeWVi9qJg1jrAxklEs9msUYR+fHwc2WwWfr9/TViitfreJEkqWRt6LbJhBHO5XbKqqmJ+fh6JRAIHDx7E4cOHi76mfmUe4EuU5DE2dvGcRUBLBfqZDpnLYfT0aWy7/XZELcWLBb8fmp3lsviYkkrh/AMPYOctt8C/SpU9VBX49vM8Pv9NEbEsALrSHbEsFeQtSxpbn7V9Ogn9dzwu4b333jOCQ2RZxtzcHBoaGlY1TcFLbATBtIPjOASDQQSDQbS2thqPExGNx+OYnJxEOp1GNptFX1+fIaLBoDcaltdiDIlEApFIpAaj8RYbRjDdIIoiMlSahBvogJ5IJIKdO3ca6x7FUJ9+EvzYGKCp+W+hlDfNalU6lMUj//tcGD8m12wqhczQkEkwY7EY3r98ueTQMrEYlHR6VQRT14F//Dce939PwHwGeWudWJZ0cA/9JkrOB84bcBywZctWXHfdTiPC8t1330UsFsPQ0JApYb6+vt626sxGwEvuYWD1BTwQCCAQCKClpQVAvgLWe++9h4aGBkiShMnJSWQyGfh8PpMlGgqFPCGi5aLruueqGNWCDXMlL4eFOTs7i56eHiOgJxaLIZlMlt4RABcKQ+f4pUm9kvnFWriAUsByDR01kcDAF7+I1ttuQyqVwuXLl8FxHA5efTW6/X5kHcegQ9f1VXPLfvEHPP7ppyLmFeSbjRCrkgimdYnY9jQoZVWaIWklJMLS7/djz549EEXRlDBPl26ziuhyTCZeEykvTfReKxJA1lRbWloMEQXyQkrcudPT00in0xBF0SSi4XDYU5+tFX1xTliPbBjBdINbwXQK6HHbE1PPZKANDkIAoIMDV02ErMPjPr+7uBZjV02FPDODS7/8Ja6IIg4cOICmpiZc6e/PX5ycfSSvDkBJJjF/6RIiK5yo/Jc/EPDozwVcUZB3w9IpI7Rw2lFUOJ0+Oc42D5OO3rUmzANLpdvi8bipiDidLL8SPSFXktW26Kx4rUiAU/qG3+9Hc3MzmpubjcdyuZzhzp2ZmVk2EdU0rabfmZe+/1rBBJNCEISigmlU6HEI6HFtoSoKlB8/AzGdBkRLh5JS51g5Ltkylkd1HchOTkD76U9x4stfNk52IRg0mlA76bUsSXj/oYew/cMfLv2Ci1QTWSorwH3/LuAbvxSQVJEXS7osLCneQ3vXK7p2SbkkM+XWXLAr3WbXzorjOFOeXzgcLnuS98ok5TXBrLUYVEs5Fq/P50NTUxOampqMx4iISpKE2dlZpFIpCIJQIKLlnD+1ysEkQU7rkQ0jmNW4ZOkKPXv27HEM6HFrYSIczhcDIP+7ccnaBffY1ZJd/LuYS9Za84D8z2kaxKEhaJIEYTGHyikP04qadXTa2kKq9pR7gaYVAX/5HI/HzghIcLC3LMlP2RR7n3nxLJZWUg527axUVTUmwaGhISSTSdMkSETUSxO/E14UzLVgYbrFTkQVRbE9f+ieonV1dY6fQy1zMNdjay9gAwmmG6yC6VShx+3+TmiXLkHX9MqzLx2sSvr/UuXx7A6hqxqkX76K1HvvIXrTTQBK1JIl+1aQWkKq9pRzgV7JAN+4vBsvjYv5MulOlmV52l02di7ZWiAIAjZt2mTyXNCT4MDAAFKplMkdV19f78nAECaYxVmO8YiiiMbGRlNpSkVRjJ6iw8PDSKVSJk8GLaKstVdpmGBS0BaiNaDHTbqAWwsz9/OfQ5+ahK5pWFzIrB6L1VmscEGx3bXkFUi/eHlJMEnSdZHJT1NVzL/7LoaffRZXFalLS1NuPdm5NPDnLwt4YW4LMgKAIMzWJFmzdFtDoeDtWB8oHfRDs1xBDnaToN2als/nA8/z8Pl8SKfTq56i4DXB9Np4auX+LIUoirY3YURER0ZGkEwmwXEc/H4/FEVBPB6vak19vbb2AjaQYLqtcaqqKt58802IomhfoacIbi1MLhhcjJBdXBksZ651WYhddJHiaXcoNati/qlvYtu9nwNQhks2l0NiaKjkdoRyBHMuA3zmFQHP9gh5sQSWzlw6z9KuKIFd85EKq/wAcAz6WUns3HGyLBtuuJ6eHqTTaQQCASOoqL6+fkWT5b0mUBvBwnSLnYiqqorx8XHMzs4aa+oADHdufX096urqXIl8PB5nLtn1QLG1JhLQk8lkcM0111RU1smNCOialheqci+WYhGydrVkfe5qIhSgAsrsNBJn/wWRG38PHMeBd7OAr+vQcznXL+O2VVcsA9z1qogXxnjIfiw1FyERsCLMlmXpugMOuP+wVtLCdIvf70ckEkEgEMBVi+3astks4vE4JEnC2NgYstmsqQvHchYQZ4JZnJWyMN0iCAKCwSA2bdqEXbt2ASgMTEsmk0Z0NzmH7FKkmEt2HaMoCgYHBzE9PY09e/ZgYWGh4hqIriYIWUby7/8evnQaOq8vBca6Cfpx6ExiF8Xjqom0w8tqCQ3zP/ocIjf+HoDFdUyHg5H91VwOs2+8AXlhAX4XJbHc3FwMJoD/+YaIn83wUAQsVeahy93l4FzurmaRsaUtzNUWTAJ9DgYCAbS2thoVZ+jap1euXDEKiBMRrWU/SK8JptfG4zUBB1DQC9guME3TNMOdS6dIkXXQ4eFhzM7OuiqL9/zzz+Oee+6Bqqo4deoU7r33XtPzXV1d+MQnPoHz58/jS1/6Ej772c8az+3cuRPRaBSCIEAURZw7d64Gn0BpNpRg0hObU0DPwMDAsp7MXDCYty41famLBi18dte0UzSs3f+LuO2JaftSOUB+fQJzE3+Exq1/l3fLOr9Ufj9dx8xvfgOptxfN119f8nVK9bZ8ewH4kwsizszxUEgRdWsUrIy8K9b2BWCvfXB4rAzWaj9Mp9qn6XQa8Xi8oKkyEdBoNFp2MIjX8h69JlDVFoNfDtxYvTzP24poMplEb28vvve97+Gdd95BJpPB+++/j+uuuw7XX389PvjBD5puWFRVxac//Wm8+OKL6OjowPHjx/HRj34Uhw4dMrZpamrCgw8+iGeeecZ2LC+//LKp6MNK4K1vbIUoFtBTqgl0TVhcS9IrqfDjlH9peb6UYGqwN1ANi7FbRbrrx/C13wgxGHQ1JJRR8aeYhfnKHIfPvi/gYnKx44g1EpZYlrQHuJQI1si48LqFWS50KytrU2USVNTf32+U/KNF1EsuxVJ4zcL0mksWyI+pEu8CyTM+duwYvvWtb+GBBx7Azp07cezYMZw/fx4/+clP8MEPftC0z9mzZ7F3717s3r0bAPCxj30Mzz77rEkw29ra0NbWhp/85CfVvbEasqEEU5IkdHV1FQ3oqVYwieXkdDeb+fnPoS2uBQC1CZC1O5BQhoVZoNs6oEuA+v8tYP7mh4BA8QVRHQB0HWomAzkWc/WaToL5vRkOf9nDYULloAW5wqo9xSxL5wI9FWK3+KmXXbhgrUE3VaarFRERpUv+0UEhdGSl1wQK8E5RB8B7Fi9Q2+bRTU1NOHLkCI4cOWK7zdjYGLZv327839HRgTNnzrh+DY7j8JGPfAQcx+Guu+7CnXfeWfW43bChBHNhYQF79uwpukZZq56YjsnBw8NQZ2cBVc8HsBBKWZv0c5rlfxtTsVKX7NJAAe0VGbo8DyUYLzG4xV2SSbz/V3+F9v/yX0puaw36kTXga6My/n4mgIRfhE6CeaypIxlUZy1WVBLPjFXo17KF6Ra65B+BXs+iqxVFIhHkcjlEIhFPCoMX8KKFWavCBW7SSuyul3JuaH79619j27ZtmJ6exoc//GF0dnbiQx/6UNljLZcNJZjbt28vmSdZqjxeKYjgOrk2uFDIiJC1WnUV4eCiFUWAczFPORpLGoBpDfyPEuBCCvQSCY7EytRcRsrSFuZ4Kov/OZzBC3oTMkHB7HqlXbEpV4fOU5GL1mnRc+n5fKUfy6MeEcyVHgO9ntXe3g4gLwSJRAJDQ0OYm5vD7OyskShP3Ll1dXWesvZWAy/eSKxk4YKOjg6MjIwY/4+OjmLbtm2uX4Ns29bWhttvvx1nz55lgrka1MrCdIILh6GTsnil1iNprBZoiX1EcXHqLxWtY3NYolWcDHA/y0IIcu7q0uo6dFWFrmlLTacdIDmvzwyP4wtKA3ojTZDVxe4tJE2E/ptU76k4baR2qKp3fbKrLUSCIKChocEo49fW1mYkysfjcVPJNno9dK2U/KsVXrQwa+mSLRUle/z4cfT09GBgYADt7e14+umn8dRTT7k6fjKZhKZpiEajSCaTeOGFF/D5z3++6nG7YUMJ5nI3kXaz/8I//AO0bBY69LwwlSOaBKdisPQ4FtMwiullsYBbHYCeA3y/yEA8Wjzox9hH15EZG0Pf3/899v7pnxbdNp3J4CFVxb+1bccM50NOlQGNB1QBUDlA5/PvL8eZ00bcFOQpJqDW53SnDZ0fd2thZjIKVFVHXd3GaypNr2HaJcrTxcPpDhx0oYXVrla0nHjVwqyVYJayMEVRxEMPPYSbb74Zqqrik5/8JA4fPoxHHnkEAHD33XdjcnISN9xwA+LxOHiex9e//nVcvHgRs7OzuP322wHk3cgf//jHccstt1Q9bjdsKMF0w3JbmFoiAV2WK3PB2u3jsJZZqnBBqUORB7gZFb6082DtXj49Ouq4fTqdxk+HevHYtijebtiEtKaC1zT4NB66BkAToGkcNE2AnvYjn4BpA+09tUshqWqeLRLgpAOqWvrLe+edWfzt3/4G589P4KqrIrj22s04cWIrDh9uxf79TfD7vWVd1JpSQT9O1YpIyb+pqSlkMhn4/X6TOzdASjWucbxoYVrzMCvFbeGC2267Dbfddpvpsbvvvtv4e8uWLRi1mUvq6+vx9ttvVz3OSthQgunWwkynnTLhS1NKcIlLFppqXkB0G/Tj5A3UzT9ui6+XrLQnAcGLpauZG8aurkPPZAqeV1UVb4704/FQBq8f3oSYwIPT0wjpPDSdg6bxUDUBuipAUQTksnlD00S182RN5tnC5ri0hZlIKHj44cv4xjfex/R0vrzYzEwcb721gMcfv4xgkEddnYA//MNr8ZnPHK/FgDxJJVGydr0gs9msIaLj4+PIZDIIBoMFIrrW8KKFWasxVZqeshbYUILphmVfw6yrM/42NLLSXMwiz5UTJVss4BYa4L+iFRUbUwCvomD2pZcQf+st1B87BlXXcHFuAo/nxnFudx2uiCJkpBAEB40ToHEcNAjQBAGKzkPR/FDUYOGRS1mPboJ8alD5x87CJMJw4cIC/uzP3sf587PI5UQAEZCIJU1ToWkKcjkFqZSM8XGpksGsGWqVVhIIBBAIBIwEdV3XDRFdWFjAyMgIZFlGKBQyrYnSaWFeCMiyUiySfjWp9jvz4mddS5hgWljONczEv/875K4u8KoKrUCZysCFeWjkYZYI+impvTogJnRwuvuh5uILGBjrxtvXhPGqMoHxNgEpQYAKCUHw8EOACh4qBGjgoUCEAhE6F4RifRWr67Ucqp6v7Q+gKOYPPJvV8PWvz+Cb35xCPJ6FrtfBnBOjYqnvWNL1q2uajiefvITTpy/gyJFWnDzZjmuucXbpemmyWq48TI7jEAwGEQwGC0r+xeNxzM/PY2hoyKhW5OWo3JUQTFJkg+dX9v178fOuBRtKMFci6EcQBGSdmikLAtRkEry1aEEFFX+UNCAvALkFgPcBwb2Ar2PpeBwHlFoicRsMJBQ3MM376jrSehY/UH+DvjAgw48ARIjwQTGEUoQKHjn4kYMPOnio1KtyRUdnoZLrsuwC7ObHaQvzrbcy+MxnNLz7bhK5XAiAH0uhvfRPBvmKC3mKldfTNB2vvz6Lv/7rczh/fhK5nIy33lrAd7/bA7+fQ12diD17ovid37kKf/EXN5V4s6vDSoo3XfJv8+bNxuunUilIkoRYLIZkMomzZ8+irq7OVGjBa+Xpaoksq/inf3oLTzzxHvbvb8QNN2zB4cOtOHCgCTt2NBSIaC2+s1qtg3qV9fvOKkQURVc9LYvtn0zaWxFGHVkK0znqsnDB9CgwNwZoWYBbbKkpXMj/9glAYBMQPAoIfOkoWTtjtRLBJPtwAMREFld/60XIH21HKhBBBkHI8ENGEDJ8i//7oEIsrXel1jHt/ncjogXbuA+3zVe90fHYY0k8+GAck5N1MDfkJD+kLFHh+eA0Nw0NpXH//T344Q9HkExmAdQBCEHXVSiKAkXRkEopmJ2dw5UrcoFgeunOfjVdjnS1oqamJqTTaRw5csQQUbpaEV3yz677xloklVLw5S+fwz//83vIZnPo70/hxRfH4PdzCIcFNDQEcOhQE7785f+E7dvra+YRkCRp3bb2AphgkNUMPgAAIABJREFUFlALC9Npf76uDhAEW6Eq/KcQTQMmJ4HxMUDN5adwAfka7jz5WwV8MSD48/LH7rSWyQPgXN58agCg66i7NAsxJUMM5B2vxLbkjCrqdq+sm2WrFqXurO5c1/s6bzg3p+PUKRk/+5mKTCaI/GVkLUlEGnTaF3KwWpiplIYnnpjBgw8OY2wsBU0LIm+t0kKsLB4vA+eq8/ZMTSWxaVMAgcDKXPJeKo1HglnoakV0yT/SwmpiYgKSlF9bjkQiJhH14nqjE/F4Dn/+52/hX/5lELIcAJA3AlRVhyyrSCQ0zMwkEYtlkE7nz6NaRe2u5+bRwAYTTDcXcC0q/ThZqFeeeQZ6NpuPJAUMndDhTpDm4sDgolgCS8YUEUwOQP7yyB+znC+3WJqJ4GJsppsAPf9+hJxCPVtIieq09rhNHSkryMfthhw0bTsef/xqxON+qCppzmldq5RR6u6HtjBffz2D++6bxhtvSJBl0pqFHItYqrnF45bn/ZicTOHBB9/Gv/zL+9A0Dbt2RXHs2Gb81m/l10N3794EQai9GKyV9VRSOJy2iug+kKOjo0gmk0a1IuLODYfDnhTR+fkc/viP38Vzz00ZYmnuXpA/n3Q9nwkQCuVniVrmYDILcx1RqoyZ28bGThSzUFOXLkFNJMw2lp2xZXNtZxRgaBZILIolD7NQ8ij8Moud/rQFWapuDXkNt3A64J9O4qpvvoH3P5dPKC7bS8qh8C6iUovT1eDd+Hqvg6pej/n5EJY+ObvAnmLlAZeUfGFBx9e+lsTjjycwP89jKaqWHIv8Tdy65k4wxe7/FhZy+MY3evHYYz2YmJCg6z4ACmIxCefOXcFjj3UhEODQ1OTH6dO34OTJ9iJjLh8vWphusesDqaqqUWiBrlZkFVE373m5biZisRxOnbqEX/xiAblcEEulssi5Ss6pfNqXIHAIh/PpH7WqI7uwsMAsTIZ77NJKNE3D4OAgrsgyotQFpcN5LctKLAnEUvmpmLP5EbFkWZKfUl9u0XQS6m/Su9kJO8HlVA0NFyYKXtHsGS3839i/onVI6jEn/atoDg8A/A2AvhP5qvDkQPQkrCNf7LZ0zqquA+PjUdx6q4JLlwSoagSFFmVpt64d6bSOb397Al//+iAGBxPQNB+AepjduipUVUEqlQHPy0il3B/fLWtZMO0QBKGgWpGiKIaIDgwMIJVKQRRFU45oKBQq+ByWo2jB5KSM3//9QZw9m4WiBGFu8UPf3C3lmAsCZ7IwV7JowVqFCWaNoS1MXdcxNTWFvr4+bNu2Da07diDDcYWiRKucA73zQEZdMkCtxW0Em91LpQ4X02paBN1MNQXH0gE+mys/E6TifMllQogCwSOA3pgv3WctDO86wpm8KR807QReeukIFIWHOUCIWADECiin2jzw0ks5PPxwCt3d8mIeaBRLImyN2E0ZY3KbcqCqGjRNh89XerL3kmAu11hEUURjYyMaGxuNx0jJv3g8junpaaTTafh8PlOOKFlPrRVDQzn8/u9P4MIFBapKLEsimMTKVJD/3slzOnieWxaXLBPMdYTbzhKVXmTEwozH47h8+TJCoRBuuOEGiIqCCaoRM7Eu3dSSHYnnXbHK4jZWTSEradZDVPLl2s3/pSxM6/5A3sJsem0EW164jP6PHLWtZAeH/wuP5rCTFbdWZNHnqSeDrUB0H4AgoHNmDxfRnhws8TfFDr4FwHUA2qCYSv4tTWJLVmVpSxXIn6eXLqn43Ofi+NWvBGSzYeRXsmmLla5gn4L1FqjUea5pOs6dm8ZXv3oWb745gT17GnDddfl10KuvbsXOnfYpCl4RzJWsquNU8i8ej0OSJExMTCCdTiOXy6G/v9+wRiutVnT5soI/+IN5vPceD00Lw7xEQP9dWL0sGBSM74gF/bhjwwmmG0q16CqGLMtIp9Po6upCZ2encfKkLl3C/I9+BL9SkJpfFE0H5jL5fpHWcA864IdQjkuWbE9+27lWjShZl+OlA394WUVoPG7Z1yrJhRLNcYuvZ6ewpVytVc3RZGceaG4HGrYCIMXgUdhFpdRypYEI+A4A/H5AWxRf+g5HJ/HOQOkV5SVyuRDuvVfDd7+rQJL8i8egRZJeD03AugYKLH7WRT6zs2fn8Hd/dxGvvjqOVCoDQMTs7ALOnJnHP//zJQQCQEODDzffvAMPPvhhY78VFUx1DmLiBfC518HxcWiB3cgF7gLEfGGD1S5D5/f70dLSYlQrSiQShliSXqLZbBahUMjkzi3VxP7tt1XcdVcaXV0CNC0As/uVtjKTWLrOljo2BINLM0St8icTiQR27NhR9XG8ChNMGyoRTE3TMDQ0hPHxcYiiiOPHj5smDD4cBqiAIuKKdQz+oRiQ8oJJu2MJKpy/RBHu9KNUKmi5QT+GlaloEBPEDUStTy5uZb9+WcbtRKk4HetjbiJseR7Y0Q5E6wHwhU2sdeRFMm13DJsBCQEgejA/eetCoYVKjMqyArNF6PpRDA3diEceqYOu2xVKIEKZQqnIWjuX7JtvSvja1wbxyiuzSCZl5PNBg8i/+bz7WFVVpFI5pNMZvPvurGn/FRFMJQ7/5D/CF/sWOH0G4PX8SS8AAeF+QAR0HxD0t8IX+l1A/X8BIby8Y3KBruvw+XxobW0tqFYkSRLm5+cxPDyMXC5XIKJkTjpzRsenPqWjry8ATSPRC7RlSQehAXY3peHw0szBXLLu2HCCWetqP7quY3p6Gn19fdiyZQtOnDiBM2fOFLwOKVpAn7b5nEUqMNZGKwYlIEcZHVZ7zO6+mTzv9sstllKio3i0LY3VNuKzCg5/7VX03XEcV7a1OryaVUgt29RiTdPt/gEe6NwKhAL5GYUWSTpgVSl1zMVB1zUALVcBfBDQOLNI0nE9ZTXGbgNwBNC3LK5TAkvfNH1g92ugtGBeuJDF/ffP4JVXriCRUJCP3CVWK229koHnv3VBWFmXLJcZR6DnjyFKrwLI2udYCQAnAGJwBltD3wbmvg29zo9s4P+Awn8J4FZnYrcTJ7paUVtbG4D8Z5hOpxGPxxGLxTA4OAhFUXDp0hbcf/8ujI4GqZslchdGu2ETKFxwh/GbrF+SMZWyaN3ABHMD4lYwJUlCV1cXgsEgrrvuOgTpNUrLhEEsTNitoVrPZWqeSSp5wSzlUCK7W4N1nKYsu+2dhuTGwnSyDzlNh5AtJwpTt7E4F1/djYVYKfUCcF1jvpGoNWOEzEUkFdLNa29tBlqaAU60t1LJzT8x/rjFx50OzvmA0BFA2AFo/ryb2HpMw7Xr/i6DnKOXLqn40pdSeOmlFJJJDvnIWqtQElM4BVqMOa7QSnUSzIWFLB577B0EAgJuuGELOjubUV9f5vqdHEPg/Xsgzr8C6LI5t0qw/E1iYMhYdRk+fAe5wA+han8FQfm/wLm+JawNbl3EHMchHA4jHA5jy5YtAID/+A8e998fwOiosBj/QKZwWhhlmG+WCpdAOA6mPq3MwnTHhhPMWliYsiyjp6cHyWQSBw4cKOguTgJ/6DWBdFcXdOqYJoFx8EImFaA/Dqi6O8G0XhaVpJXY4Vas7Z4QkzIaL04itmsbCmW1yKsW+5oqEUs79y15bCuA64KAj1u6WadFk6xXunE6iBywpxVoCgPg7a1UEqjqLq4HCDcCDTsALmKO1LWrl+CqrsHS568om/DVr7bg9dd5pFIk0Z22IknkLvFDW63W/LFE0Vry0SyYyWQO3/52Hx5++G2Mji5A11X4fEAwyKOjI4xrr23DyZPb8J//8w5s3158wg1c/CLEyVfzb9gaMk6G4af+tnz3OgeAS0EN3IeM+ApCmUcgYBNWikrF6UfPCrj3L4KYHOeodDQ6Th5YuhuzC98z/08LZq3yMCVJMqXerDc2nGC6walaj6ZpGB4extjYGHbv3o1Dhw7ZCjARXFow53/2M+RmZiBoeZuOXrt0kg0eedEsFQbitH850a30cawRt6WOU0xsfYksrn3oVfT+7jGQhJolC5L8bxHQcsSylDFVLByXB7AbwDE+n4NjFTYy98hYErdir9XAAwcbgHox74KlBY0WYLcuWJ4Htm0H6psAXSwcF/mfaFlZ1AM4gVTqAH7+c+KKoxdXaauSrIM6v3nrdUD3B33iiXE88kg/hoYWoGkCSF6oLKuQZQUXL2Zw8eIAfvCDXnzxizfh7ruPOb6OOPpjiEM/BJTFnqu0dUl+06myRde5dajiGcyH70B9+hH49f2Or1tLKmnt9eQPRPz1FwOYmeOg+5B/n7rNDwBYr6cCF1b+JxIxW5gsD7M0TDBtsFqYuq5jZmYGvb292Lx5M06cOFH0bsyueIEYiZjWMHWgsGiB5abwUjwf7FOKwtWJPOUG61j3J/+Td8rZPO+0rzEeTQcvuzHNbO6IK0kPceuNFAAcA3AYS0ms1iBDcqPuhnYA1/iBkMWqpMVSBimyUnqM9X5gdwsQpKJqyZonrWvl1TUAuAAQPIB8EYbgkmvXcOnyWMrgzSG/DkYP1v4Dsa5hJhLAww/P4dFHpzA8nISm8TAXUKAt2HwlI11XbduWGUPPzMD//gPgMvGlB2krkvw4HcJ085T/RwOg8uOYCt+F5vRXEdZucHz9WqFpWlnW3De+68P/8w9+zCW4vIuZPrfo88I4D4r5jpYEM5mcx7lz5xCNRpFIJJBdLNtZzdozE8x1hluXrCznQ/ATiQS6urrg9/sL1imL7W916fKhkNGpxK0rNK3kBbOcpXj6ZtONhelkVdJUE3fDqTqaLk6h/bU+9N90yFX+JcfpyJfFswT/2P1tHajTwem/fQBuALAHhWJJl24tdbNCjnkQwNUcEOAKhZduWuJG2AQAu4LAjob8+UIHPVqDhdy31gR4AYjuBOo68qktVguYWKxEw0zrqcVc6MQlm992bk7F6dMpfOMbjZidTULTQih089ILwkuKz3GccRw7fBf/F/j5y4CqLt29WQWTDmx3OJTd8oXKXcF46F60Ze5DvfpbjmOoBeWkudx/2od/+JYfUpbLp9dazwVyY1dwT1poUdIpJRwH7N9/Fa699lpIkoQrV65gYmICAwMDRmlAEp1rV63IiVwuV3FO6VpgwwmmG0RRhCRJuHjxIiRJQmdnZ8E6ZTHsLEwimDrHmU1LO68JGcfiNeU+M88yDhfblF7pcB8lS6DFlwPgk7LY/PYo+m86ZLyCxYFn/GW+Lku4aJ0szGKPBQF8EMB25N+YNSCHiIbTWqD1+B8A0InChiW0uGVhlwJZSB2A6wSgdXHWdxqb2/VUQvMmoGUPIIaWLEprcSEy6SbLPPYishzC5z8PfO97KmZmeGhaM5aqOtAfKnkDKZjvIPLftVMheH7uIsSBnwCZ9NJ3wC0emvzPY+nu0s38zpkXBzROwnDwy2jP/Dka1d92cYDKUFW1ZMqaogJ/9XAA3/qRiJTKASEUei7IjZitWDpd1UuP19f7jWpFPp8PBw8ehM/nM6oVSZKEvr4+pNNpiKJo1NeNRqMIBoO2bngvFd1fDphgWtA0DbOzs5iYmMDBgwdx8ODBsl0UVgtTUxRkRkdNamDy/jncwPOcewuRPiZ5vNw1TLtjk5t4t8exu1w4RUVwPr9wV0z7Sr5GMSvTjXs2DOC3ALQg/6bowEIyGdlOQA7H/h0AO2FeT6KtNhIs5IYOAMcBRPjCY9FWYDnBQhEB2NsO1AXzA6atVXqMxNArWyh1APXQtN/Ga6914he/CCA/s9tF1pLB2/mQF9eyuULXLkG8/BS4hRFAXbx9tAv2sdNazvy7+JICB5VLoS/4AHZmBLSoJ4tsXTmlLMx4CvizfwrgX1/xQeZgFku7QK/84G0oWNw0/uc4IBpd8l3RgUhO1YpIyb/JyUlkMhn4/X7DCgVgbO9mvnz++edxzz33QFVVnDp1Cvfee6/p+a6uLnziE5/A+fPn8aUvfQmf/exnXe+7nGw4wSz2Zc7MzKCnpwf19fVobW3Ftm3bKnoNOwtz8umnwSWTS629XCCSyaDEDnb3k0BpoSvmhqWPWWxZqNixyP9CTsX1D7+Ktz92EyZ3X+XCu2pxxbpV62LbRQHcBKAZS5aJ3QTkxm3qB/DbAK7CkljSrjJ6fdEN1wI48v+z9+5Bchz3necnq6q7p7vnBQwwwAADAhiCeBEPviBivZJ27T1bEuKCsXR4Ixg+W6Hg2RJlaa1br2PPG3t7ew5v3Gn3rDhLRzm4q5Apyz6fbJ/CYniX0pq2tPJDEiiRFAgQ7zcweAPznulXVd4fVdWdlZX16JkRteTgFzHTXVWZv8quysxv/r75y1/iU2763JRatlwesEH5dhRhtOLvIq6Cox66NtVaTaZgcQag+AR4G8F1aHnB6E7a+LUl7IzDUcgcZqSP3kP3tgWQE+eZPfYKA40FIlN/KmCqUTqyRmUx8IxmaIkFTvV8Dq9mM+y+J0HZ4iXNS/b6hOBXPl/i2z90aFl06oQ+gAop/tgrSprwkZE/f4PtDmBKKVNBvFgsMjQ0xNDQUPtcvV5vg+hv/dZv8cYbb7CwsMBv/uZv8sQTT/DEE0+wbt064+//xCc+wSuvvMLo6CgHDhzgqaeeYvfu3e00q1ev5nOf+xxf+9rXus77o5T/9jZ0+zHI7Owsr732GteuXeOxxx5j69ateN5iidC4hWk5DqJQ8OlYXfTbKHXcUSzMvM42qiwmBqx5PLrIAOzKOSEldsT5R2pes3S+B787Eoh9KabyauAD+AAXBqxR/8pkR6oPpQf4GXwaNgzbGv6VFZ15aGQL+AfB3+pAXzn4VP8KKfp03ZuAD1mwrwirLD/uQBX/M/zrC8q7mNZf6YeRA7D+p2BgC/QXYED4/jzhvcLVKe0fCXlm7oUQMUr2L/9ylv/zf/pPlGYuYEsZBXx1ILBo6TzYTo30QfPNnt/luv36UpQbJcnCPHXd4mc/08M3Tzu0ivjvvkqnXoT1S9+D3QiaOm0VbdnqTiWQzyrUpVQqsWbNGsbGxnjxxRf55je/yaZNm9i3bx/f+c53+PCHP8y3vvWtWL5XX32Vbdu2MTY2RrFY5JlnnuGll16KpBkeHubAgQMx6jpP3h+lrGgLs9FocO7cOaanp9mxY0d7/VC9Xl/yJtL1enQ0bfX0RJb4gU/Fto9T5jDzit6GugVME4ETSjdFMbXf0sQc+/70+4z/660JqZcw95Fmha7DpzrDDSVUGjbseGsp+VWp0qF0Q8tftwjDKbssqeAD5aimS52jClnMPFIOyraFztysaQ60SbeboPjlW9ULI6NQrPhrQXVHIZV9DacoMyX6zoXoOA+98kqTf//vp3n99QXe+th/oMdrdCIsaZZiauXU0xJvJyb3GICWqPN6+T+wb+E5NrnJS126Fd3ClBL+/KjNr/2/JW5PWsiQgtUpdA8za5EJmDr/I7EsKJc7YLQcc4+zs7OsWbOGp59+mqeffjox3fj4OJs2bWofj46Ocvjw4Vz3WEre5ZAVB5jgj/CuXLnC1atX2bp1Kzt37owAaTeh8UxiomTt3l5czcmgXUUT6qqdYw5Tl7B5QPfOOiZdoYSMVzfNKjKf2nB58Fsn4F8npzdZnAkJ8z2YB/A3Bwm93NURi+o0kaYrvNaPD3Aq8OZ1FtL1r8Z3FhomPu+ply+PbMEfFPQr+nSwbBKnX7PGKjawrgBj66CnAFJEgVKfB63jr0LJFDN573nwgx8M8JnPwNGjgmazwofG/obB4ixCevGQhOou6mjXsnn/REpWPdekwffKX6JZazHWOpD1w3KJamE2Xfjfv1nkhb8pMNcUnWUjuvOYS5eDkKShr3/OB0y/+18uR528S0pM98tr4S4l73LIigNMKSWvvfYag4ODPPnkk8bFupZlLSslK13XD42n7YUJGmhqFwsBMISWaDcOQKqFmQV0Wb801JXnvuq58E8AwpOU785RnKtRq/pLc9LnL6WZjs0Llg/izw32Eh1kq2CS17IcwPesHaDzMHVLNW+UnfX4YDmo6NE7x7yBCGz8edkddGg6XV9YNkn6FKL6Ei1gM7Dbgb6CeQ5UBcrQUSqbdU244AB7qdf389nPDuE3PYklGjy+/jT9hTksNW6CWqk9ci0liUlKJ6uXsEWT7/R8hflakz2tn8h5g2QJAfPMPcGv/ZcS37ng0LTxWQd97jqsDyrTkEUHxazL+HnLElQqhXZ5livKT19fX2a60dFRrly50j6+evVqbn+RpeRdDllxgCmE4PHHH0+d4F7qiEW3MG/9+Z8zfeQIBc3qVKnZ2Em6o2RNTjdLiQGrp+lmiYpJl5Cw9uwtDvxv/x9nXYvpPWNM7tlGc88uqCQBqKZJB0udbgu/78S3LMvEKdM0astklQwA78e33lRnIRWc8joLPQC8B38OUX1YenCDPORGH/Df4Yf1U8ule8A2c5YtlK34A43V+BalGlFI78QXWML8ofR3DunZBvYD4FbAtfBCYPQEo303+dijv4djtdpZIvQ15N+SR68jsYPoy9e5DheXv+v5z0w0Znlv46cRS5hUrzU9/vBsmc98v8z1aQvZg+8jlfX+dKBMNCb1E/GEti3a23u93XthHjhwgDNnznDhwgU2btzIV77yFf7oj/4o1z2Wknc5ZMUBJvgW4FIsyDz6I04/PT3IViu6tZfhUz/IA5gmgAqPs5pAEkiaZj3y6korj5ASUWvS/71z9P3NSUYQeJbN7JaN3H14O3f27OLMoQ/g81KLEAHswY/eE6rQwc0jm9oK+8JVwN/DB48QlFQrVZIf4Lbig2WF6ANWAThcy58lo/iWZTiY151gVAov78zCGnxadz2d36rPU6rWTldAqdQIIaDaB6vGoDgIsmDeDMWF9279W0pOjQgvk1Rpu8Avfy/waAadnjVhkofHa8XvMW7d42drP0tRCSkyOVnn618/x+7da9i+fVVkflCVH04I/udLu3hzttd/1VWiAKm+xxpmsNQLHjmflrBz3bKgWvW7/+WMI5tnvbrjODz//PN84AMfwHVdnn32WR5++GFeeOEFAJ577jlu3LjBE088wfT0NJZl8Tu/8zscP36c/v5+Y963S1YkYOaVxYaJ0i1MO9yphA5BEqnfwYfQ6rgt/P4lkXnJkCwL0wTYSd8XC76q2PUmj778Bq/0VWkV/OchPEn1/DiV89fZ+Od/zfjjjzO/fqC75SQAJelH3HkYn8vWl3mExyogJU9d+WB5EH+eEeJWah6AC3Vtxge4pCUC6jKBNLHwBwSP4lskOuCquvI69lTxKeIH6QTkMXXgoQNSWK2TKmVSRSgI2NALw+uhUKQdRF53GFJA+b/f+ucMFKf9yE9Jor/DLM9qYRqwCu3T9FN8xRK46Fzmc9Uv88z8U4zKYX74w9v8yq98i1On7tDTY1Gt2oyNDfDoo8M8+eQIu3atQWwY4nMXS3xt3GFaVDv0q8mCr+Nb76m0a8r5SAKzSWpZy29hdhMW79ChQxw6dChy7rnnnmt/X79+PVevXs2d9+2S+4CZILZtL5rb1y1Mu1wGg552NU6o+OGykryAqTcLQXferSZaV2W+lkMqc3WcpkQ4Nq1yiVZPjx+2LRCrpU2G5XDeoOIh/l4d1gikY4FrgSdAWgq4iWSPU11vPz7ArSX6UFVgyhvqbjO+s1BByavPMdZTflsoBfy1mruJhvNTBwYhNRwPoBMXO9D1CD5ohmVJsnSSCJmsyrkW2FGANas6of706EIqaAb7je8bOMLeVUexxCKZoKT5b6XY+TPp1qhkRszz+erX4PVR/uAfzzB9rwGUaTZdZmZcbty4y/deu8cLh6dxfq4P5/Eyc03bZz4KmJ916MGs7vmcNKpNo3QSf2HnnG135jCXK/D69PQ0GzduXLKe/5ZlRQJmN1t8LQYwdQvTqlQQtp2/bktABJSsSO+PdBX6PfKWPsmyBL8t56koSWAbipBQnl6gVGjSdGXgqyFwSwVa5R5a5R76Ll3l9uhDwZUcQR4GWhT/3jys9pCWQHoW0hNIGXx3BTQL4BqehKka9OE7+IRBTnSACy1LHSxNusbwrdQwtKaJis2zbKQHP/btVjoRilSrJAQ7lcJLk0F8elilX1XgVa3xrndBwbd+x/ABuR+QThQY9Tm60CJWsHFN6Q4bK9fyA2YaW5BocWaZox0x4ZYHLDx2hw9e7+HNz/Vz6nPASBm5bwieHMXbuQ56y7jSpq7vva0OEsL54LRg/yZr02wKY0bU6N+PYg5zdnY2l9PPO1lWJGDmEdu2abVaiwokbFlWxP159q238Fw3soQ7Nng0NIiCYmFmiQms8gBdXut1Mc3J1GQLrRYFCc12A5VY9SbFepPi5AyP//v/wPmXfjKqKIGetdfU6X3vJLIqkVh4roXnCaQMQRPcloPbyhmVoM+D93qwSvhxCcNdQtTdQvJaltvwQU6ds1RBN9SVJRV80N1IlG5Q9bXID2x7gf3Ely+oQBaGXOvGWQh8Gvs9+MAeOrGEOsNg9iFYhMtcwPgc/vm2z9BjL6Q5s3bE6NCTkCbzYlSZGszAdBOJwClIdv5zweZ/XmZa9nJXVrndEtQb8zRrLm7LwWs60Cj4TEdLdJbhZIUkTDIUk8DTaIrGE/b0dFr0cjr9dBNz+50o9wEzQZa6FlOVm9/4Bs2JCWwlLF77U5IYS1Z1+kkDtbRr4dKSrIFr0vk2JZsTuU2DATWb44GFh+VJpGUhLRFx8bcbKb10uy+TlDfN0PfYBFRk0KlZwbO08DwL1xM0G0XcvK+w4mG/dwHW+fgoXQvpWQFYCn/OrWn5HV5iuYLPB1wfLPuDztUjoIgxz6Umlkn6IfhGAsX6vGcIlnmMsB78wAahlRpapToV2+3emhY+mD+Mb7FaGB14YtZURplHStcpiEW2v5zAqYNhp66aHH90oIzqCb+VRJNVYppyscl8cYZab4marFCr91BfKNGslZBzJXBF5z3oStWBFZgbUyIlpFuT8V8Csk3HArGiXrqPAAAgAElEQVS9excr7/atvWCFAmY3lOxyiLNImqLbwAWmdtXNC04D1TTv/fTZkqhYEmzPw5ICGdBtEt+ik5ZFaWICZ2EeIXqCPkpG72x7DDx4l1Xb70AxNP4s/09YvqVpCZpegWYblAylUUGu5FI6OIvY6PlgKQNKV1pINwDhpg2tUjyv9lDEcAvrHzSQfbRp4c58avC9KbIBs89D/EQDORKAdpi/TeuKfFuQAayXvmPPUFBY0zKbcC41r7Vj4VvRe/AtS0EUJE20a9pOKEqf/ovrvsyWysV8dGxWAzFcl8H5JEYzrR2kg2bnWmifWkiE8BCWF/fqSxpVmq7FvAUT8iWUOvILBJGwePc3j84vKxIw88hyAGboZetolUinYo1sSiAJux0Z9Zm+L+YFm5x/8jgP6fl0HQIfMDvjdxGcl+BJhOex6swl9v4/X+V7v/DLMQ2202Rk5xV610+BA17QJXltjTYugroote+X2KMEfZsoePT+/UmcBxo+dkgb6Qk81/IBzxG4roVs2Zl9k7WqReEfzkFfYPVK0aaGpQz01YrQdNI7+qpL4X0LsN4L9FhIj2B+NgDOlgWtDBpNSMTOFuKJJl5PEBIndIgKvVSliG5sbRL9h4/hOwuFUY/0EHmqRZlCuyZJxZqnx6rlo2N1ScsTXrNMoKfXFl2Rel1of2oawyhKN/qSrEQTe9rNaNSoyHxT1cJ0XXdZ9rDMu6zknSwrEjDfDgszjBZk2zZOby8IERkkquxLCJqmUoX76eZgQ43tK+0FZ7VF9XtWXNq85bPVAyk7dGyb1ZLYrWb0HNDTM8fY5lP0VBfAUsHSB0wPixY2XrA2Lk9faxVdBh69Q2G0BnZI0bUCcAPPs2i1HDy3GCtPTFe1RfUnJxCrpG9QeuF8agc43aaNm+acAli9Lj3vm0Ks9fBENH9o9XotC6+VsVbVljjvmcfZ08Bz/Lk4PAvPC8A7dIhqFH2qOVMkrJNwQARLbUTc01OPTDPfyZqmVpU+a5pfGv5iJ1jBYqULsDWBpF5knSWNp9UBNeFekYZPHMuSADXpONbo1cwmbtff2qtS6fQMy7UO8/4c5gqWpQKm6mUbbh6tS+LUg6onZ8NPamd5vVtDZi5J72J2PjEdW2jMVNiDBB9Wq8Xqsxewm02aFBDCY+3ANcaGz+IUGsGUoIVLQL9i4yFwcXADODaWU3OItAouqx++QWV0FhyJ55Nn/qfwPz3LxtW9a01TmAWX/oP3KKyvIVWQkwLPE3jSptW0cTMsQqvcov8n7mKvb4It8ML8gWXpOzVZNFrFVD2i4FF9zySF7TWkFeqw8QIQRwo8V9BqFvBMlp/+Avtc7PfVYcxD2n4ZcG3fWnXD7cNEx6LMYlKTAAEoFhsMV25iL8U7Vr8embdMK1LcqkwaVJrAU4MlZAoA5yxQ/uNMiWbo7Y1amMtBydbrdXp6Fhl05B0i9wEzQUw7jnQj4dKS5swMMydOtM8nTlfopl0wf+nYSpoESWvUebxk84jVdQM130OPkx2jflstdv3Zf+Y7/+NzTGzcwo6hY6yp3sSyZNua9GeGaINlgyJe8EujXZ4kdlKAsDzW7rhG36ZJcDodHIi2fheLejtcUPKPtxyX1U/coLRxHqwgfwC4UgqwBa7n0WpmgGXBZfDRW5RGagHdHFC4VuCpKcH1HGoL6dSZVXAZPHiT0gM1pC2U5xUAuBS4LZtGvZTuBSsBS1LaNUvp0QVkBVwsPFcgPdu3nkOvZNdB1gr4e2HmEL0R9OA7I22F/2HmD+m7Ou0n6IaSzXL0iWGhTqUmYVLGQDeBulWPJcEcdHhdHzCYqFjzzcz0biS/alnqSjqJe3vNm0cvVsJVAW9nIPQfh6xIwHw7KNkwv+N53PrmN3Hr9XaM6Ej1DepwUpec18JMLEfG9by0bJ64tGl6w6aaV89meZ5HVl+kZNV9q00BMy+Yq2zh0FKeao7xPAAbH7rE4IY7YBO1LOncxyPdigMfeAd33qZv4xTCke384Z8rfMvXE0FnlPLDh/bfoPeBKaRNZ0AgOmXzhI2XAUjCaTH8xDjljXNIq6MntMalsHCl5XuTZYhdaTL4E7cobqrh2eGzsfCkhRtau9Ki1SjQrIMrcizd0UFyDN9xKHQacmH3X5+g6s4jwoq7lHnMlLwdjNGtw7SZb6GlT573NLcr37pPLFiWtZnLGjWhsXrNp2SXGzBDuQ+YK1SWbYuvUomW60KwpMREnRrreyCFRczFqPfIE5s6CdxUUixv8HX9WCfWssbjSChPT3Dw8EvcefT9gQ4rsPo69Ks/X5lCweo3EAAeD2w+z9DwLSwrfB8dgPPalmUOBwgBQ5tvMfTgjcBK7azZC7/bWNTaYXkSfrSQrN15jcEH7iIs1Qaw2rq8iLVrFstpMrrvIpV1c4EzbcdidhCAFThEZVNmlbVTDD9yHbvaQloSGTx3DwtXWEjLwnVsWm4Bt9lFF2JJrAebyB0uco0dRP6x/HWJLcHIxDU21y/Gn1eSZMwHJ0lnI3ff6jPjkQqOIiGNDpZC+557w7qEghq+mzqQSKI8mSQDA506vhxzmIsNI/pOk/uAmSDLAZi3bt3izp07CMfxKSAlmEFql6BcLOSgZNVsepPpxjI00cXd6AnzmrxsdV1JIgB7ocG6Pz3M/HOPByAZdth2AB7QMkJ4x8rU264lXDatO8+GdZexba8NAFEL00q2LDUGb9Wa26zbfBXHaQUOXaG+Dvi2KGQ8M8mqdbdZv/UaouBFvH7VMjUztDhOk83bz9A7NN0GXdUZSiJ8OjXDahbCY+3ma6wbu4YoSB8cFceqsGwuNnVKinVP6gu3ehuU989S3F7H6xG4noXbsvFcC9e18FwH2RI8MHmOAzdfw5Fu/JknFpooVunXTGlTEyYlsmJpOkAqlLZnHhL60y4Jk6lJGBd+mkbapsZuVGJO3N+/vBbm3Nwcvb29S9LxTpAVCZg/akp2ZmaGGzduUKlUOHDgAN/p76dm2AuzbX0lzWXgW5h5wDLJYSfvC04pApDfSzaL4oU8S1QkdrNBlXlc7KCz9zvpejvqeNZdaPd9ltVi0+AVtqw+F4ClfzFK84q2FSd0HZpUK9M8MHaOYk+jTRfrfw2KmRuFDA7eZfOD57CdFhh0+GVKt3Ydu8HWTadYNXS37ekbgqQ6J7uQYaHaVpOxbacZXHsH0dajA7hNkwJSeYNp7EjP4CyDj9yhvHEeWZB4wn+XLjZS2rSkhefZuJ5Ds+HQLybJt7C0C0mstOm8bR4W1ETF+tcMFmm2iZoMlqa05oIkJIgmFgL6+jr1St3QerEyPT39rg+LBysUMMEHzbSdxhcDmM1mk7NnzzI9Pc26devo7e2lWCxiV6vtNCZHbw/F+FSLJDrRfvJamLqkAV0WSKrp8u58kiZhGkukZxAulI7fZvVfvcXcP9pGnaKRkjQFLdMVW3hs7TvNWN85REDDdqzJEJwsmpjm4OKFLDo1djxwjErPPAgZWG+hY42vs5WjWZVLs2zdfIZKcS4IvhDVEVLQaSKEx9YNZ1g/dANhe3SoXNXitWhqlqX+q4pOjV0bjzDQP4FlhWBJ27p0FbD0n1P65GClPMOGbZd9i7cggwFPSKn7tLcnbDxh07Ac6l4Pbsvm3/z5/0FffRaRPX2sPYgu00eLqyjpULCmG8TPd3SY25ACyjIc+Yp4w0uyIs0FjeZNLY1Jmb9TiRq4AJY+97gSghbACgbMLNEDqKeJlJLx8XEuXrzI1q1b2blzJ+Pj47RaLeauXPHnaiwLcuoLReADZga+xMujfOaxDNNG0aHkXKmX61we8kcsNCmdus38P3owxx0SzgvYXTnO5tJlLMsLUlmBDRPSixaNSHR0U2H8a5Zw2bHhLQYqk0G/J4Jn3AHeFjauqVkFXs8IScmusWvDUfoqk8H2bercpz932aSQCZjb1pxk0+pLCNvfdTnZQk2uARVnjkeGf0BfedKPthQ8E91CrVNsDyri1pOvv2A32LLxDOvXjmPZHlKIGFiG35sUqFNCKM/KcZsI2aWFmdXP69OKyjndQozCjJnrNYFm3Ibz88etUAU4s6zHNCtTf0QxM9aUsTNR4u9Usrxd/30Lc4VL3hHX5OQkJ0+eZHBwkCeffJJCwe9UwmUpd19/ncljx7BbreTBY0of4YRtNgUx0waf3RAtaW22m3WYWfcIrdVUgG1JrFv+yndDXxfrxlQJr+0vvskDzmVs4VOepnk9mTivF31btmixfdVJRvquIgLLUl3i0pm3NIOc6j/50NBxVlfuIITXtgjV+TAXO5PO3dR7kbHVZyjYzaAPjQZy8ME7vXn32rM8MfwafcXpEM3blmX4fEJwS/tFAo/hnmvs2XA0sJg7QBn9E7gUqNETK9uTJ19lePo2IoX1SRWzz40vakxmwxRHOusZ5y2SLM34XKj/KWUXA940K9PTrqVamBg+/e96aLzlkOnp6fsW5rtZsijZLKnX65w+fZparcaePXtiE97hbiflSsW4F2abmg2KEBkkKsUK98TMKzr2Lhbo9KbWrfOQ6Vx4Ppd7Qcuj9/eOMvXhR6iPjeS8c6i/xX5xko2Fq9iYvGF9QKkZaV790y/1A+XLbOk7j2OFA58o9ekm0rpR2dZ3mk19l9p69LnC0LEpTYYKd9jf/wYFuxGxTmkDuKBBwQiY4TuoiDkO9H2fVYUphPDaVqXqBGUr3rlJfbhDk729R9nScxHbCvVYbcBUaeoGJTykXsUBeOjaOdZN38IScmkjs0THHnNC3cqMv/mO6KAZBc6orpilGYJmGp6l7Tmadi12wmxZqjeq1Wap1WoUCoUlz1/CfUr2viSI53lcvnyZ8fFxHnzwQdatW2e0Rh3H8b3PlCg/8fBayUDZ1qPtWJI2H2lqh3kA00Tk6EVZjKWaNBjOrUuCaHYafDoZ5ndNBRrs4Sgj4gYWXrvzV63L0IM1yVbQz40419hdfoseqx78ljB+bQeE8yxFGXLusKP3JCW7TpQ+9e/qYdHIeFsVZnm89zX6rBlAtkFWBV6fyk0G7wJ1Hi+/wbBzGwjrSYeCJQA804BClX4m2V85ymprok1Rq4ESwrLpjkIEv1cV222BlB3v5uWgM3SJUbEdSWJp4vRs3IqMA2mCbZfnxvrNk7z5jPn1FmdKKLFtC2hw6tQparUajUaDCxcu0N/fT39/f5sl60buA+a7XPJQrkKIiAfZ3bt3OX36NGvWrOHgwYOprtjtwAXK5tFJEht5KpJ3x5Ik/d0AZlp8kLyWalqTVcuUqUOCdXeBgT8+ztz/uiGi2cS+AZSZYw9vsZoJBDIAj878ooRgSURPQsmIne8T0+wrH6PfnglsozDKUNRzNEtXkRqPVV+nz5pBBEDXDgbQBuD05ligwd7yUdY4d9sw6+tw2+Drz6Emvy2HJo/aP2SDuA4sPnDDCNfYax2lKuaCckQHJB2Ho0IAmPoT6YBNdWGOX/va5yi2muSIF5EuedjSdjnMzj1qOeMWqImeTR4CRvJLRU+WlZnUEDMlaZjaOXYci23btjA2Nsj8/DynTp2iWq0yMTHBpUuXaLVaVKvVNoD29vZmLjuZnp5meHg4TwHf0bJiATOPhKDnui6nTp3C8zz2799PpVLJzBs6DdkJlGwosbagNYpC6Csg0y3MhOxdA12SqCHtsgbFWXpzr/hqulT/5krC3aKyign28Ba9zLbBsjMfZ7UjA6l0p6kLVG2GEjUecX7IKmsy0OmniETzgUwq1qbFw8WTDDn3NKALaT1BM8lZSCnjdnGGTdZVbNzI/GlI+3nYiXOoAA4tHuYtHhBX2q5P6mAidMhpxFArWoM2c4E9vEVZ1AJgBBks/wkBxEWwQDn1uQBUmWWvPMqgN9Uuk+GWZjEbfPnykWxV6hKv2/mcgdQb+n4KomM1qtajiXoyFVA9l2TGxk7oCaNesp7nUSwWGR4ebgOelJK5uTmmp6e5fv06s7OzAPT29rZBtFqtRoyO2dlZtm3blvIM3h1yHzBTxLZtzp8/z71799i+fTtr1qzJnbdtYQ4MICwr0ujU9hFaU0ZmhSgl242E98gCp7xTI0udwwzPS7Ip2XZ+D5xb89hTNRjo0INR69JjE5fZxQkKtFAp2I4FJ7CCAAjdMH37OMYG6wYEABynHONLP0z6t4iLbHUuYuOGXVbwKYKfaZP1pjZxmR3WaYrtfbI686dhWcIAB0nP/yHOsJ0z7XJ0qFzV8Sjdp3orF9jHUYo0InlCKtdD0MTBUyhqU3mK1NnNMTZzmQcvnsfJvdO3QbJ4dcP1KF9htiT1tPp13UpVuY/E55jLSsyQVEo2DpD6Od9L1h/kmQKvCyHo7e2lt7eXDRs2tNPNzs4yPT3NpUuXmJubw3EcJicnuXjxIrdu3crlJfuNb3yDT33qU7iuyy/90i/xG7/xG9GfJiWf+tSnePnll6lUKnzpS1/iscceA2DLli309fX5O0A5Dj/4wQ+yn9Uyy4oFzCxK9tatW0xMTFAulzl48GDXE+OhhXnm93+fxsxMNmmnWpBLBEy1DXU7h2m6lgd4s/oBvUx5REgonZtg9VffYu7Zvx/TWKTGwxxjA9fbNKffcdvt7/68YBEv8a4m8s1jO6fZLC7iEK5vjIKDb4mZ5y3V5z3EbXaKk1RYwAde3ckntMqSpY9p9nCcKrOoFq4V3E1i0ciwcjdxmZ2cpCfYoNL/TS1U56V6Ch8qkDzAZR7jhxRoRgYl6jIU38M3uVsRSEa5wl6OUWEeEDz+7SNUp+a6X38ZVZx9LoENTVeqAqp5QBI/Nvl1J4jJGEyyNtP+YgqTS2nbHQszb5Qf27YZGBiIbN/VbDY5fvw4f/VXf8V3v/tdvvWtb7F9+3be85738FM/9VO8//3vj+hwXZdPfOITvPLKK4yOjnLgwAGeeuopdu/e3U7z9a9/nTNnznDmzBkOHz7Mxz/+cQ4fPty+/q1vfasrw2W5ZcUCZpLMzc1x8uRJCoUCw8PDrFu3blFeZOF+mI3JSVoLCzhSplblJErWFgRr9bIbuL5FV2jNLdYyVO+ZV0+eAXRXQbikh9XUJ3Ykm7jCFi5QDTZd7ACRP6/oe2k6NCkY9i5OL+UI19jDcRyabWAzWZdmfZ0y9rDAPo4ywLRyNXQK8fU0KWQsIZHs5yhD3EO05yotdM/aNCefASbZz5v0aoAblgNo08GmOiqBUa7wGEdwaIJiIat0blPZYs2EVgWa7OcNHuAKNl577rbYSnB1WozjTx4v2dhOJVGHHmOknthx9Ea6tRnq8ZeVBM9bdo3YycCYmjeJt/X/HEdQLPrvaSlh8QqFAvv372f//v185CMf4Td/8zcZHBzk1Vdf5caNG7H0r776Ktu2bWNsbAyAZ555hpdeeikCmC+99BIf/vCHEUJw8OBBJicnuX79OiMj3XnK/6jkPmAG0mq1OHfuHPfu3WPnzp2sWrWKM2fOLDo8XmjBFrUNVU1VGTB70NFZVpLHCjTdR+2+8liBKmWsSrprQ/5yQf49OgFEzWXjZ77DtZ99hMbaAqu5y3bO0Mc0FgQWjt22+jpLM2zqCNKAxCSrucM+jlJWgFida8yyxFTZySnWcQsLl9ALVgf2rPnPHZxiA+M4bTo36p3rEQZ4N4tNi0d5nSEmIPAaNgFuWkiNEa7xCEeoBlayHlvWP063LPuZ5FGOsIoJoDPAqd6cY82lu0Rqyo/CQzbUm2nwhQniXr2qRzOxz7gl2jnWCxG5oakQnc+ur+sgGf8M6VhYvs2jZ2ZmWLVqFRs3bmR0dNSYZnx8nE2bNrWPR0dHI9ZjUprx8XFGRkYQQvAzP/MzCCH42Mc+xkc/+tEll7tbWbGAGQKalJLr169z4cIFNm3axMGDB9vXlhqAHcCpVkGYR6ztMV/SgFAElGyKO3yWLNbpx1TObppV0mA4tHq7EbvWYMvCOdbyJr3MBOBvt0O2hbFm/eUi/trBZoZzj0l6mWM/bzLIVJA6aon5uZOaTFT3BsbZwiVKCgUa0p8Sgl1R0p/oKiZ4mBPBvKUaDcjX5Q9szIArg/97eYv13NScfKL0sr58RP0lvUzzKD9sW8nx5TB+sAYdtFVLbC23eYw36GUmuNbxDB64O83Q+QmEP6ZYXjGsBOnEPzdblHEF/qfJwOuAY3zCNI3CjUkaxqV9jx2bmQ79s1Tq1OHl2jx6ZmYmQteaxLTuXZ8aS0vzd3/3d2zYsIFbt27x0z/90+zcuTNG+/6oZcUCJvgv+cSJE1SrVQ4cOECxGLUclg0wLSu1zkMCaOJ7yeaxDpN05wXLrMEs5Ac6XZ/+PQ0mYuWXULo9x74XX6H4bzYEwdiddhQaN/B8DecUPYoxYgzMM0qqvVCixj6OsoZ7iPb6zbgV5cb0xJ9ahTke4Ugwb6l2yC5qZJ80cWiyl6P0MQtIpRyqxZvefNdzne2cphDsFK1buDKgUZOkhwUO8H1Wa1ZhNLqRQCbSsDDMTQ7wA8osGOdue9w6llSeRregaXLuSXL4aV/vAFkaUCa1OXWtpamuG28c7oOp07JRxcmNRj/W514iX5IzC0EkLJ7ruotad6nLwsIC5XK6V/To6ChXrnS83q9evdp2KsqTJvwcHh7m6aef5tVXX33bAXPpIR7eoTI/P8/x48fZuXMnDz/8cAwsoRN8YLEihMAqlRBWZ9+9tIGkSdTg63koVYhSqhbpL7kbqzWPx21yJxPV0w2QC0/iHJlkmFsMcZdV3GOAafqYo8o8BZpYqb8k+8nt5Rgj3KBEDYcmDk2KNCnSoBSsJtTFDL4eezjGIFPB3iqNYCViuELSjS6fSCjPg5xjPTdxaAZkp1+GIk1KNHEyZj5LLLCPY5SZD2L+hL+nSYkGRRpBOZLfxD6Oso5b7d9RokEPjeDYL0OY2zQgHOIO7+dvGGCqfd8SdXqoU6ZGiTpbXrmM1dTaWJ55SD19F5JuKUbB0jSwSTbyoojd1mvCrqS/tBthSBdLk22OqpTscm0eLaXM9PU4cOAAZ86c4cKFCzQaDb7yla/w1FNPRdI89dRTfPnLX0ZKyfe+9z0GBgYYGRlhbm6OmRmfoZibm+Mv/uIv2LNnz5LL3a2sWAuzWq3ynve8J9VbdqkWpuM4iFIJaVmE+2EaGRdpGCQGkjdwQUyncpy+ui/f+dAyzDOfmnWt20DuXhMaR1o4x2ax91Ta3qH+HKCTkCsuZmNEspejbOEidgBk6hxj6AWatZA/lE1cYZSrFGgGpenQsKEu884rHVnNXXZxkh5qEYtQDZqQtsLfpsVuTjDEvSDaUfhkQgsXBFZkCBB9cpIxzjPGhcDxKWpZhsHm9ehGqo7V3OU9vNq2sj1FRzgXbGOx+tQkdsNbWsCCpAZieOHtMnaxO0caG6CCpdSOI/OeaWxpuuLOcVIDj53UTdHOeSGgt3d55zDzhhh1HIfnn3+eD3zgA7iuy7PPPsvDDz/MCy+8AMBzzz3HoUOHePnll9m2bRuVSoUXX3wRgJs3b/L000+3y/zzP//zfPCDH1xSuRcjKxYwIc6f67JUwLSk5OhnPoNXrxNyrontJgBNU4nsHAiTVGU98s09msqlR6FcTLMylSvPVmF6s3fvQP0tl+IePaU6E2XWlkSwCTwe4BLbOBcAnDpnKZC08GPOlpQ8yb+sn8lguUQHJFRPUh8wVWSI6yhSZw/H23O1aqcb0phpNCrARq4yxgUKtDQPXz/8QtzLN/rkVjHBo/xQmX+NOhllbWFWZp4neI1VTLZ/ZQiU4VZoPoVe8OcuTY/DMP9olCT/m0znnqgCPaSdXpz4PGd87lIqn6balmgx6p96IbyEa4miAqWq3Bedkl3qHGYImHmipx06dIhDhw5Fzj333HPt70IIPv/5z8fyjY2NceTIkSWVczlkxQLmj3oTaYBCTw/S83BdFx0AjFU5YeTpdOH0Y0rjkH8eNCk8Xqgn733TLNfFAK87D3N/1qT4szLF8bW7ofsoV9nDscADNb5hskwBBr32FGmwl7cYYCoCdOruH1lAB7CFi6zjZkC5Rh19bCUwQJJUmWUXp+mhFsBjx8M3BN20NZsFGuzjTSrMBTmsSBkIwC7puRRp8Bg/YB23EdD+/WHsXZtwfWyJoR9M0HOnlvlMlkskaC8uOvzRrcM4KEZHrrp1qX6PDoaDczKhz0myHNPmOOKFzplY0tvbGQAuByW7sLCQK/rZu0FWLGACmTuWhDuOLFZs28Yu+ZUzCVB0KzNyImhfTja2G9tceC5vc0gDuW70mPKrP6ubtaGqktphF1mLAqZJTx6XjlEus4Mz9FALUkfXR4YBD6L3SB4WbOEiI9yIAV2HPk53sAEYZJJtnA2oWIiDlYeXsQxlFycZ4g4W0qhDZKzZ3MFpRrmGpcSnjf75O5IkvcFdnGAj19vuUZ11mmEoQZsWRUBQvl6jNNVEpMdlT5a0dDoda7A6TTVEtzxBb1u6w1aSRWkCVBFtnGkmbeKelwkFb19IiwjtU7J9fcs7h7lS9sKEFQ6YWbIcc5iWMvJKbCfBCVMbEHQf7UdvLlkvWQ94oOvIqycpn/59UTufeODdltS+Wsf6SDlXn5qUZgNX2c0JKizELMtO9Jr8TWM193iIs5RZUMBJD3+XPUm3m+MMMtl2YIp6tcbD8Omyjpts5jI91JV61hkARD1a47KGO2xPGESEc5CtlN8xyhW2cZaSkj8K2P6zCFuUcIlWkK5HUcSB0fSpSJyDMDv1xDEpyfXWrD8CuMljt06mpHiypJwzF9RwIQqa/f0dC3M55jBXyk4lcB8wU2WpgGnbNlZPj+/0Y/C2NVVpYzms7DThdZP/ZTcvOc3KzNKjMknhsak9L8rCBGQTmt9tUvpI/M7mPjIO3+u4zh6O0xOAZWeOsBPFp04hN2AWaLCfIwEV69twKg2pb+Sc9Ls3Ms4IN9reuBKfglXnL9MBU7KPo1SYD6jYUDo60qIKFamzlwbcblUAACAASURBVKNK8Hr1qfo6WliG/H6aKrM8zmv0MqeUufNMQ9AWgXVr1T3Wfu8ewpPJbtOLqSRLluS5St3SNEloSaqDBKlelNr3NIA0jT4TrdKk1h/P1N/fGfR4nrcsgHnfwlwBkkXJWpa1pE2mVQszrX3EREvYLSWrHy/2Jetl7iaebNrCCSvHhKrxcgta36hTeLMB+/xRcj5L0++u13ODRzmCQyuwquMB1V2cdqeeTG11ZDtnWMvttlWo0rAdCtL0Bjp6bJrs5wgV/K2yOl6x0c2vk0WyjbOs5p4S4L2z4CME3DTZwgU2cCPIL+h4tfrl6Hi4mu+/l2MMMNX2wVWdp8INthsU2/XJangMHp3CcmW3wZiSJY+3LAQesipHGx9uSeU4ydJU30vi+1EaUbsrMW3hRco5TU8+MaOrEIK+vg5gSilz+XOkyfT09H0L874sXWzb9rf3EtFRq8q+tGccwgZlaBCFAKm6he5Q3XIsK4EAuBc/fmjrzWNhGm8jQU5JGG/BvnzEGPhuKlu4yg7OtOPDqlZlCJ6NnJZleN813OYhzgaReCTRjZxD3VYkT/y7yw7Ot8EmSuOGll16mcrMs4djlAMqVnVaku1weMlPq58ptnOGIjWEAnRJjkL6u9nEFbZyqT0QiVrtBI4+xUhe4WZUpMVy7qa5ywxRn7n+nExgGR+cmsA3p2QBocmijHQcpgxJ3yWWFV2HuRxyn5K9L8siU2++yZ033jDSsbp4iQdQTPBIN4mpueSde0xji/yOb5FAp523uwTdSDlmQP7LKfjQYPuKGrFH77aKNNjBSR7gKnawJlGN3BOujXSNQdqTxabZ3nGjYwuG4OR3onlizg4ywUOcjVCxYacdesWmUbEWLts5TS/zARms04E+4CbVQIkf87afmXbMW/U3QNRRSH91Bers5a1g55HQKzcasF4vvwT6jsxgNYJS5QS2lIfQhXXZKYP/mQR08WUjqkjjuQ7FHz8fpE+iW3WRhr+kNImZwnMddFV3KlkuyRMW790iKxow81IRi6UtCqUSrelpbM8zVuN8hcw/hxmmCZtIKOGykrQ84Wcalbr0eCC+ZFmYacvOpAQx3kJcaCK2lkh+opJ+JnmYE/QzjYD2GsCOQ47fsTcoBVQsCbr08rs8zImAiu1s/4VizXmITAwo0GAHp6kyF4BuFKh8AEqnUldzly1cbAdKiOYP10ya35zED133AJeC/S39PHo5krZGk1Kym5OslvewhRIgQURD55kAf/jb9yhOt6KVaqnAqeoxHQuQIm5FmhJGYUYfkunzmvGaEwFP2flM/YFZI1bT90RFpnYRtTCXg44Fn5IdHBxcsp53gqxowMwjYXi8xSzuLfb1gRIWT5dIVU5AqtBLNg8bmgR2eZ11sq7lDWmXlD9sxovyklV1TULvZ+9Q/51eY+qirDMmzrOZi8Eay6inqBuAZguHljaBZmb5ok9oA9fYxnmF3tUdXPytxeK/Ivp9A9fYxDgFGkoZ1SAB6U+rQIOdnA7izaLk78yoqoCr98E2LXZyimrbUSga5KCzdZhZBr0JxrzzPtgKOlMPIhg4CIEU5tpn6+Hw3g4R6jNIAr+EjJHrccvTZHHmEh3XshpjYnrTkDyeQQjo6eluL8wsmZmZYfPmzUvW806QFQ2Y3QQvWAxg9vT1+R6ydKqtHn9DQtsZIFb3wzKEbTqHiRm3sxbnlWoCurxB05OIoVC6LU+sj3DB/uo8o++9yrmnx9oFK7sLbG1dYj3X6LFqYAeWggi2/xJ229vUDXY3SZKk6bEqs+zkJOVgqyvdIlPXb6bprTDPNs4FAQa8mJ48lO4IN1jPDQq0FEuoo6Oz5tEsm7jKSDtIQvhbOveXkLB1mMCSLo/U36JfzoOQeKH1JvA3GxACT1jBu4k+TWvepXdqHmGqz4s1eJZonUYtw/i1PAxpVFc08g9q0IJkUiR+3oSBpryxBCaFEtsWVKv+IHE5AfP+HOZ9AZa2tKQYbu0VxJFVxdgmElpiuCdmN6KqyaJk9fSmYwAhu9OTNNa1utRjktZdWPvtaQ4u/JDJfVWcVZIK89gOyAJgg2uDZ0PLFkjHwrUsXMuhLoqZaxpDUctp4bKHY6zhLurSC3XNpT/flz2yGeMca7hLZ9utcClISAeml69EjYeCNZPhWCqyhKEdXccsPSzwEGeD5TXhL5URPWkOUOuadxhp3KEggvIL9c/FtQRe0RyRYOjNSXq/N5/O/y+3GB5FlkWZJ13c6sxwADI1ChPO6edMu5NECpbU2ogcW5ZoW5jLtRfmfS/Z+9KWpQBmT3+/T0spcWRTu9IkwLSiSUzdYNpgtZtIP/ogVrWIsyjZPKNvMnTkFVmHub+Env2w+tgcVIE+/09UgRJYDlCAYlHiFVzqZWnoxPVfkGxdbuMco8Fmzn5qdc2l1d6XM0tWcY/NXA6s1DC/Toem69nEFYa4095YWrUuCcqVJlu4xFpuYwc7juhl8GJrLjtAIKTHnvlTFN2mv6cAdGKZh6DpJNeCtbcnfOsyB2OyaImyrbGX2vGKVc+pnzoR37kWvx519onhmQwekqd5HJlubLpmKqRJT6op6p+zLNGOJXvfwuxeVjRg5qFkFxsez/M8rt25gwfYAWgmAVG8gUV1dWth6vrSKNCkTsEkb1fw9Vz6JLjj0DoLzl7ABeqB4hZQDv4kfui1SDg9869Om8lazV22c0qLgqOuubSoGxcUxvXtCLxSO0HVonRsVgi9HhbYwRkqQQi96NwnmK3LznFVzrKZy5RoAF4w9+i1dyGRkEoHb5y/Tv/CjG9Xi+D5BuDU9mtxzL++Oj1P3+RcOljmpVd1MEwCSEO6KA7FbxgfOulpog5CnTZnupmC3kbrMOE4KW2qJA8BQAZesh1K9u3aPPrdIisaMPPIYizMe/fucfLkSZzxcUQKFRsey+CLkWEhf/D1JIYnj2WYFnQ9lG6ALq2s3QBvWpnkAjTfBHu3oVzaQzBZNOm/pZPYocEjHKEffz8+33rTt+3K15RGuM4I15Xtv3QqNBswt3CJAabay2T0BfS+V2yyjs3eJVYx4cd7FYAUbe9RCchYcNeo7Ji8gONqS0KCP3VwYqrra69NUPhPLWhqRVzsHGQSHZAgyTiVrUhNq1qaJr3qOWNbSBotJwGl/t2oXL8Qr/S2/aOxMO8D5n0BugPMer3OqVOnaDabPPLII5w5coTGxARWsKwkSVSrSRI0QRUwQ6RSr6fp0c4txrvVpGcplqEq3fSNqc/Ng+ZrUPrHAQ2blCl1pGHiwXwR+J6kuzjNGm5jtVN3FvSHAOXl8EX2dZ0I1iyqwNsBvSw9JRaCIANNpWtUHUySl5EA9HqzbGldpkCjHfBGihAo/bfSspIpjU3T1yg3/J1QjBaeZs13fr2gPF1j4M4M4i4+I5D1yJajsum6rDSnHhNNq6+r1C1K03flfcjooDh1BGiKAJQElGF6abqY1AAkxaKFHewXuFxzmHNzc1Sr1eyE7wJZ0YCZ10u22WymppFScvnyZa5evcq2bdsYHh5GCIFTLreXlUD62C9yrNX3pQRfh6VZhrql2m1+/XwIvHl0ZA4yJDAH3stg/aIhU04qK+3ZbOESY1zEptXuPKNzhdlB0UPZygWGuIsT6PKLGF3ukgaYAo8HuRCsKw33tQz3tvT1ZZVlU+Mqq5rT2MK3Lj0BnhAdhx1DBxrWTcdtMTp5A7vlJtOgsYcZnPAkA9dmKNRcEqModCtJjGmKz41JSbyaJNGvcTEPUnUPWaI+f3oFTwoBm3TD5DGeplz982/iOL5FWK1Wl42SlVIuC/C+E2RFA2YecRyHhYWFxOuTk5OcOHGCoaEhDh48GKk4hWoVYVn+HJE2h4nyvX0+oRHYgrZzRU4MiOZfRJ5QdKDL6oNMI2+078qUV2a/YNIVSecCR4HrwNaMwimSRLapMiKvsYsTlEXonKPubBI6yeQLM1Zmnoc4p2zKHJbCnz/0gS9dV1nOs12cUaICRanYLKejklfnwdplSqHTkvANLhms75AC6pFlINGnNDx5l975uc5ZHaQsYzZAUJhvser6LOIKUNPS5cOmZMmZv/2WRfzGnRpgOhdan0nKow8iuY0q+XWwTBvo6SCZCpb6X/R6uWxz5coVZmdnabVaVCoVCoUC/f39lEqlrgMZLCXW9jtR7gNmhiRRso1Gg9OnT7OwsMDevXvp7dUX0YNTqSAty1i/9bovTfU84F9Dp59uqmY4psxLyZpAXG/LeeZC1fxqOdQ03QRxTzvXvjAF/A1RwDQqSQ9hrkqfnGaPe4x+Me1ntZQ1jiLcEDq++4dZv2QnJ1nFJCKIBhRuqKw66qRGIZeSHfI0ZTnvRxcSYSdO4LST3FHLIP9o7Qa97rxPLXeytgvuRUz/6C8pNJusmbyH03Tb9TLylzGaWn1uFmfWhRPABOmVoJs+exEA69fPzkBDVxZ9jlEATXINC6/pjKlPy4bmO8nbeKXRTkkjRpl4oCkISiKgv7/C7t27ATh79iyWZTE3N8f169ep1+v09PTQ39/PwMAAfX19uS3Q5YgY9E6QFQ2Y3QQuCEVKyfj4OJcuXWJsbIz169cn6nHKZf+atqwkaaAYoWSVi47SGWVZZKaYzHnXYSblDyVr7jHx92jHi21aSc9PusAbwE8CDy1SuVKuHq/Go7UjDDKJEJ6/KN8S7cGPFALPsmmKAubeTEb0rZb3eEBcpdi2LsNOV6Vi04nqPjnDA+5l31lIQCeqjt8Ze0IgRDIK2dJlx9QFf91nwFgItV6pFqLhqfTNzjIwO4vwIqejgJnQmxSmW1RuLPhUbJPo49Irw1L6XRMlq1VaabAuVZGxzKa323lYuQaxmQ0+p6TSLkm8VfR6uWzjum57p6bBwUFWr17tp5CShYUFpqenuX37NufPn8fzPHp7e9sgWq1WI/1dCLIrRbqcHXv3SRZoqoA5PT3Nq6++yszMDE8++SQjIyOp+QuVCgQUbdq6Y6kemPR0aWHqTSdv7NYsXUuZw1QlC3iT9KVStLPAX0IsgrqSMH5PQ2k9j/2zxxmuTdBTdyk2JKWGpFT3KNZaFBoudo5g+qF+W7Z42HuLijePI11s6WLLFrYMA/O12sEL0mR3/RS9jRpOy8VpuVitFrbrl8XyXISXrmPr9GWKzQaiiQ9aTfxn1fC/i5TswvNYd+suTsP1l+y4wZ+nfU+Q8uUazrznbxid99HlqSDLlaZt7etDHZ2e1Yl8c01eEkmpV3IT2Jo6k0gm07FECKhWi0gpaTabzMz4Xt+tVgvXdZFSUqlUWL9+Pdu3b+fxxx/n8ccfZ+PGjXiex6VLl/j+97/P66+/ztmzZ3n55Zc5ceJErr0wv/GNb7Bjxw62bdvGpz/96fjPlpJf/dVfZdu2bezbt4/XX389d963U1a0hZlHQqefEydOMD09za5du3Iv0nWqVUSCE4X+XULiHGY3wddN9wlLkDVnmHYuL7WbVZawHHn05Pm9bSrbBY4Ffz+RnkefegvF9lz2TpxnpH4Ty/I7GKyO9SWD+b66LWL6dHIvlA3uddZ5t3FECwRtj9RwXtsTAhLirYYy1LrHuuYdCtJFCNle1ieFn9+1wC0kj31LrTrrZu/4QJ9khdnJzF/v1AL9E/M+qJoo2CC/ScS8pOdGA6spYQ74O8zgqleK5Wb4Ivrjb8wENdGMqrUZLWhnLtn0cDWFaaO+8NgUPzO9kBmJOwn7+0u4rsuxY8dYu3YtAwMDfj0MBlxuMBgUQmAFDov9/f2RPq/RaDA9Pc13v/td/vZv/5YLFy7wcz/3czz55JM8+eSTHDx4kGKxs47XdV0+8YlP8MorrzA6OsqBAwd46qmn2tQwwNe//nXOnDnDmTNnOHz4MB//+Mc5fPhwrrxvp6x4wEzbRFpKyZ07d5icnGRkZISdO3d2xdW/9Qd/gFur4ZDeXvTveoxNW3Sa3mJGr8u1HET16eimHKa0eTaR7lrqwNeB7cCoVoD2A5bGh+G4LbbcvcrI7G3sMAZgAAwhcAob3JJJaVwEvpPN7topKtTBCj1SQwD2462qXqlJ7+jBhUtUWjWssGIoNKolJDJjh/F1M3eo1hc69UqnKg3Z1V82fOUeVtMLKqdBhzr1qk4dCHDuuRRvB5ZpDR80l/u9q5LOuKZkiJ+TxjRJwElwPrRWg09pAtRIhpwjw4RPYwJjxGpAYlkt3nzzTXbt2hVbO+l5Xhs8Pc9rg2f4aVkWQgiKxSJr1qzht37rt3jjjTf44he/yL/6V/+Kw4cP88d//Mfs3r2bNWvWtPW++uqrbNu2jbGxMQCeeeYZXnrppQjovfTSS3z4wx9GCMHBgweZnJzk+vXrXLx4MTPv2ykrHjCTZHZ2lhMnTlAul6lUKoyOjmZn0mTu1i3cWq27KYyExmPnJM91JgcWtzuI6dxyBC7Qrd5lFQncAL4GfDKtINGTdqvFphs3WTNzD0d4UQtK/Z69vaVyC8nW2mUGm7PYgVXYDk4OeJZLy7HalH2SrK5PsmbhHrb0ohF1oD0XmdSKJVBsNlk7PYGjOq7pgJnSC5Qna5RnGlitKFi3P5N8lYL1jr1v1LEa0i+MPn+5nKJXzKSKqgJ6rDj+RdUvx9SeVEkz+BJ/qilhmvWYaVkmnYye6+93eOihIk888YTRmSe0KENPf9XqTLJCp6am6OvrY/v27Wzfvp1f/MVfjOkdHx9n06ZN7ePR0VEOHz6cmWZ8fDxX3rdT7gOmJq7rcu7cOe7evcuuXbsYHBzkO9/5zqJ0lfr7284ZSVMTqNdSWmieaD/6fULmyyI/aIY69D0p5SJ0pMlSJs9TdXv4y0z+C/BUegaBxKm32HjpBoOzc1hSdh5WCE4WProHYKlGDEpyGwHobc4xNn8ZR3ptKzUEOQRYnh8cPklC3ZtmrtHTqPvHCtCJUGdGC+6bnaE/dNYRhr+MF9F7Zx6n1ow+Qx1wwfgwnOse1oyk7Up8legcZhI3n9dKzEqnlS90VA2l01byDgNNwdY7VmU0jWKJhqMkfccSM2uaj4KNNbJkhbYt2LKlh09/+gAf+tCe3CxZCKCWspZctUIXFhb43d/9XUqlUpIKvyQGBk8vQ1KaPHnfTlnxgBk+fCklt27d4uzZs4yOjnLw4MElv5iCYTLcVOclxLf4Cg+CImSwbomS16JTQTIU/bjb4Ou6TvV8t0tL0o6l+kXid9DfwKdl/348owCQHqXJGsNnp+mZb/jWWwiOIZCEf4n9gRmJbc9lbOYyPfUA6ALwFQoIq0s4kp5pf22G4fm7WJ5npkJTrMuweBvu3PGddUx5IdVqlp7HqvFZrBDwTFN0SYWXEvuGh7UgOybbEXxa9kflVJkHONsn4iZpko2mAmBcYcaoQQXLLJPVBJCmtInWZjxDsQjve1+RX//1LWzZMsD09DR9fX0REOxGwnznzp3jox/9KL/wC7/AJz/5ydQ8o6OjXLlypX189epVNmzYkCtNo9HIzPt2yooHTID5+XlOnDhBoVDgiSeeMI6YFrM7eamvr21hgnmNcuQeCecF+aL9mGkm/77d7GWZlCbLIEnLqzdlJdpf7jy5KS/wvT//EJ8ufF/0kqi5DJydo/faAnZoeakWZfjnkAKWyTI8f5f1s3ewwodmokAjeuNvQEiPDTO3fM9WMINURusdnJyhOjvve6dGjSBflEnpyLMM6vnguTl/7WTS70irDC6U3vIQDSW9bl2aZDmNh4QpQ5OYQTF6HE3TeRjx+qlZm2lWov6pUzymghqBNKpYCBgaEnzyk2v4p/90P81mnampKa5evcrs7CyWZbWXigwMDNDT05Orf5NS8id/8id89rOf5Qtf+AIHDhzIzHPgwAHOnDnDhQsX2LhxI1/5ylf4oz/6o0iap556iueff55nnnmGw4cPMzAwwMjICGvXrs3M+3bKigfMa9euce7cOXbu3Nlej6SLZVl4ntd1+CenUkGkBC6I1fsU5FqMp6yaNsMQyaVH0h0tm6Y/j540tipXhjo+aE4APwdiStJzvknf9QZWI7D4bDqWZfip/nUppWad0YkbFMJwiqqnVB6gCaRcW2Dd1B0stePsUsfw9QnsupdMoab0j1bDo3KjDi3ZeZ7qvQW+dWoAcgnYp0DUiVZw05KSxQKkCcBNeg3Rh3RwTBqIqQCoD9zSQNJPr8ag1UckhpslnTOBaSxd9IJtC3btkvy7f7eZf/gPHwCgp6dIX19f2xej1WoxNTXF9PQ0169fp1arUS6X2wDa398fm+ecn5/nX/yLf8HMzAz/9b/+VwYHBxN+VFQcx+H555/nAx/4AK7r8uyzz/Lwww/zwgsvAPDcc89x6NAhXn75ZbZt20alUuHFF19MzfvjkhUPmGvWrGHt2rWpFEW4FrNbwCwEgAnxRpbEvEhza+yaktVVhExjGuDkAePlWovZLRYl6ZyS0HBhrUzoe13gFbC+Cz2bgSHpW53BJtNtoHSC7wV862+RXkkjd2/7c4amDlvk0ys8j3X37lCqNaN5k0BCAapQKlM1KtMLvrOObiGaJrW1h1e816I42US0tDRhJUqrjw2wbuHnDQBfXgNuEPMAj+hNKU9uyRgIhOI3s3jCuDer0NqriLZXpbBJTAjgR/uJ3jyaYVk205ZUq/DBD9b57d/eyfBwckB0x3EYGhpiaGgoKJ+kVqsxOTnJ7du3OXv2LFJKzp49y+3bt9m6dSu//du/zbPPPsvHP/7xrindQ4cOcejQoci55557rv1dCMHnP//53Hl/XLLiAbNUKmXuRhICZtbkti6FSsVfPqCcyw1YWsJCzg48bHv6Pbt90THGJ5ClxqVVATwtXd5rVz14awF6GrCuBsMNWCdhuKj8ZonvoXkV39qs0tkvs4gPkOGn+qBym7a+9M7OsfbuPexW0PupVG8SUBmkVG8wfG8C0ZJxgAxBN5UykPTdmcWpGTxjw+9pYWulpHy1gV3TrEu9DElSA/sikWcmF0DMkOs5diV5gFUbXMjYxYQRiEGSYgaEwBkb70plbtTUqLJN1zjAGsLrCSEZHm7x679e57nndncNaEIIyuUy5XKZkZER/zaeR6FQ4K//+q/54he/SKFQ4M/+7M+4fv06Tz/9NI8//nhX93g3yIoHzMWEx8sj0vM4/qd/itfyN1tKaiuJJ7SLRWHWkapPkW5etKd9V9tpHks1S7qhdpPyq3qmJcy6MLUA5xvgTIO4Dhv64dBeLaOH7xQUzq2FyyLSzINYAQxdrusxeuUGxYVmNLa3+pcvTjsbr93CabSilqGuL0VES7L6muaso+c1lSW4Zt/zKF9sdOYfk0A76f7H8CMKqWl0Wtdw38TjxBspnzmfTfQ1C+2aMH5PvznEdieJ3Eef+8RMr5JwbAJOLW2hAPv2TfO5z0keeWQso9z5ZWFhgf/4H/8jrVaL119/nYGBAW7evMnhw4ep1WrZCt6FsuIBM48sBjCFZTF54QJuo9F+yPoAUf0uZawfVpR1v8VXeL9QZVZfrQ9ak7C7G+/WpH4Buqt4af2KBbgSGtKfssQDIaBgQX0SJuZhle6VmTlyyRa9Ox26PkV1YsHf3VK3LFVgzpDK9AJ99+Y71qUeUUeQ8vD8hKsvzeDUXP+l6kAH0dGKAReKl1qIuvTpbJV+Dcujb/ysygIwiZ9XrSymNZhpmLRYSjZJT/vTRMOab5wMmtHlJamDWJ2/zTMw0y3OFBq7t9fln/yTcf7tv+1ncHARHmoJcvz4cT72sY/x0Y9+lF/+5V9uW6zr1q3jqaeeysj97pX7gJlDFgOY4AdfV71kdTGBkomWFfiAmdeyM+FBnhedR3cewDS1c113nj0x8wzAwy4t9CcJ10m6EhoenLoJB82+XOabph0nSGmmwdpLk9gNjUJVadicQQ+Grk5SqLX832Gat0x6kaF1uODSe20Bmkrl0QE35eGLusS54WGFAKeXI8vlegrEDdCxpv4XUGqSHAVwOeYxdXDMkTgr2Ho0ck9ch5rWi5wPdynxFUpTA08DxKTGE6QXFoxunOdf/8vb/PzPr122tYlSSv7gD/6AL3zhC/ze7/0e+/fvXxa97xZZ8YCZl5J1cwfc7khB2YU8CUhUkXpCVZdQ0mSIPkCFxVl0pjbbLfAm6VquSD/622vfL+ikXr8MD2+Bvh/R2j+rJRk8N4Uz3+oUKC/IaVKerFOdrCGa0gx0OXT13GhQmGnFnXVCSzXjwdu3JIVb0rcIY9YZ2WB0An/0opczbRscVRbb72eBbaTsQmtm6VanmiYPVZv5M7Osy6TRYoDITlHy5P7bPP9pyfaHhrPulltmZ2f5Z//sn2FZFt/+9reNWxaudFnxu5XkkcVamMVqtb2dUHxuI4GhSUBUO4xpmiF6hJ5Q0rb4ystIeixthKWWLS9gZlnnWetC55tw7IrhQp6bJNy0/RylpPfSAr03av6cnbqTR0s5zql/4Oosznyrs6uHqi+HHtGQlK/XsUI6Vf1rkSs0XfG456fzlL+sMoQPpE6HjtXFtC2Y+vk2iNTulcyAmCdEF7M7iUrdplbmJJBUOwgBlXKTj/zjE/y7/+UcpeIEk5OT7ZB1S5Fjx47xoQ99iJ/8yZ/ky1/+8n2wTJAVb2HmEcdxmJ+f7zpfIdg7Lql/NlmCUk8UlqGLjsXULhe7HERvx4v1bu22PEaK2nCcFd/W9eDsOOx5EPryTvGkUWTKtcKtFv2na1g1jf5UQ+ulUbHKpHVxokn1Vs2nQk3WJWQ+NGfapedmsBQkj3WoH0+CNUHUOlXTZlmoV4G7xH6zewtiO22b9KcdJ6VPy2eYv5XGUac6qNVB0nTOsM4ysdBh8ALDC0ijYrRrwoYtI7P89ifv8jPv38z8/DxTU1Ncu3aNU6dOIYRor6HsJhCB53n8/u//Pi+++CJf+tKX2Lt3b2aelSwrHjDzVCrbthdnYQaxZEPJQ9UkpXFEp7mpc3d5JLTo0vKYaAmpuwAAIABJREFU2qvJhb4bKjUJ9EI9eX9DojWeQ4cEpmbh3GV4JO9cZg4Rs5K+o3WcGQ0soQOWeVuXJ+m9VMOZ0xx1VMehHA++eqSBWNDKo/blGTqsW2DNEn3xaqXL4qOuYLSUmifBmya3l3BuScOrtIqhbSIdX6URjxmbTN2qc5zRnUkkwgyUHYXx52U4LhYk/2D7Df7vX3PYNOJTsL29vfT29rJx40agE4hgamqqHYigUqlEAhHo68hnZmb41V/9VXp6evj2t79NtZq8bvO++LLiARNI3eILFk/JloI95NQ2YApqjvIZgqaInMwffD3puBuLLnF/WoK51JRCJDDKMX3LAbxhH541EHAlvHUGdu2CUqWLG+sFaB9Lym+2KNzw4hHuVQvT1LoMD8iZcqmM16Jh5HQAzpx79HAmXZ8aDvOrerIiF7lg/RDzIl5Vn0nCa9cwL8DvhnpYTtGtS6KfJtGBM/16/FpqAVQFpg7A8NwHq01+5b1X+Y1fXJsaOMUUiGBhYYGpqSlu3rzJmTNnkFJy/PhxFhYWGB0d5dOf/jSf+tSn+MhHPvJjDWj+TpL7gJlDFj2HqW00nTRnYmwzWgNylOCraRamriv8W8rmz6ruvHryUKnd6tCv5TF6wrS1Ohw9Co+/V52DzJHZUCDnLXDOembHGoLPck6dLUn5XBN7TilMEiWbIoUrHtasVNyFtfxZOq7jB0bPQoMk9vEo/pISwwuRWf5yP4b5TLNzjhp4IOp7YOJ6pZJHvx4B5zxIrYoE25JsWzvNZ//JNO97dH3OjEoJhKBSqVCpVNqBCFzXxXVdvvCFL/D8889TKpX46le/yvj4OE8//fSPNeTcO0XuA2YOWTRgGiL9qGICyiRDt5t1mKZRcJ69LPOMnLuxVJOsaC+jPN1Y0Xn36PQknD8Lm7fB2q05MiQV4ho4R/DjpOogGQKTvvA2RewJSfl8s6NPp2KtHHrqksJF6VuooXRDx3rAzeA3dbPnZ6i7BczQebGq6jlonge7Gx59MZIBulK7ZgI5Y572tXy8b2xes90ItGP1Bhrt1FP0+ODoNf6vn+9heHVnI+alyuzsLC+88P+z9+bxUdXX///z3pnJZJJMEgIkZCFACJDIKvvmUinuioKySaEggiKiglYslZ/aqlhx+4qfWlq32qqttkJLEUVqSIIKRgGFsBpDFkICZF9nuff3x+Te3JnMmkxAcV48hsnc5f1+3ztz7+ue8z7ndV4hOjqa/Px8TCYTxcXFfPHFF9TW1gatnwsZIcKk61yyhshIVXzdkydGC1W8wM1KvUBbTUUv8GTFdsTCdG0rUEvV3YN1R1yy3hDIMVmtcOALmJwEBndpJt58vwAVIO4BlPgvrftV+dv1ivL0hQDYZIyHbIiNOLsNtFahHwSmOwG6KhzWpSeXrrcT1QjCIfzXM3XljhbgOI5jcDl+uQVsRaBTKpT5+sKCRaZe2vH8YOiODF3nMN1bpq6dtnmKhLZ3b5am7Li+e5paWDX2JHdOiUffEbUSD9i7dy/Lly9n5cqVzJs3T3XBpqamkpqaGrR+LnSE0kr8QKeECzSajp7csG7hEomgCBd43cdN+8rfgZT38kbqHSW6jlqGvo7V33aUtmrPwnd5HejoDLAHqMRBEBYc6RcWHK5MZVkAbl7xLISdkNvSR1zTSPxJSbGD/kBr33bap4L4Q4In8B3F6grtST+BZ7IONP/STxe0xzZc99csc89TrqIEbaTo7HJVtm/bzh2JOluuLgPy8vCkF2WGR1fxj5vPsPyqXkEjS0mS2LhxIytXruTtt9/mF7/4RafmK4uLi/nZz35GZmYmgwcP5sUXXwSgsrKSqVOnMmDAAKZOnUpVVZXb/bdt28agQYNIT09n3bp1HR7H+UKIMP1AR39gkk6H5MZydfXKqJ+93Fj0nbmR4F+UrKdiz9p7niB33LWrhT9t+DL4/GnHdXtJgqJvoLrIjx0VVAK7gFIc83TNmlcLDuIMlHBsYNgPQhPt8zeVd39OZDEINbTlWVpaX0oeqC/CtAMVrft0FKfx+LDgTe7R6b0j8HRN+GzTHau2wdP1qSVRd206a9E64PN5QYYIncTMXsV8OBfG9A9eKHd1dTULFiwgPz+frKwsMjMzO92mXq/n2Wef5dChQ3zxxRe8/PLL5Ofns27dOqZMmcKxY8eYMmWKWzK02+3cfffdfPjhh+Tn5/POO++Qn5/f6TGdS4QIk44ToidIkkRBQQHF5eWIBkcsvTurzfWzhJc5TAEl5icQIyZgV6qvtoOWhykETrzu2g/EwlQgWeHoFmgo99CoFmXA/4BTtFmRinXZQhtpBoqTIJa17mvDWejAH6KjdbtiN6SrFU7w1U4lUETgPywFZ2hLJ3GDulyHW7YdvD0ABvKFBkqaTmle3ucjHdt4+1twetB0tkgFdXrFbck+GQRZppeunt8k7GP1kDpsLY0d8mS5w1dffcW1117LjBkz+OMf/4jJ5G8UmnckJiYycuRIAMxmM5mZmZSWlrJ582YWLFgAwIIFC9i0aVO7fffs2UN6ejppaWmEhYUxe/ZsNm/eHJRxnSuE5jCDjKqqKg4dOkRCQgJDRoygUK/3yyvlejEqmqjYAdFBMK7buWtDaxFqr9POzj0q6Igr1c29IuBC1O7u5+q0n+BmpZf2ZKClHr57HzIWQ5i7IEQbDpm3g7RZj8p8pVJDUwnw6UBxT/FrHHOhykFoX34KtdMAgtad6s6t6audAtpyPzuCehyk77p/62d7Zev4/AkmCnQM/hClm21kNxt4thzbN+SuOol2O8/rUH+ABkFmtOEMz4+wkh7Xl+rqaioqKtQ6lNHR0WoOZUREhN8P9ZIk8corr/D+++/z97//nUGDBvm1X0dQWFjI3r17GTduHOXl5Wo0bmJiIhUVFe22Ly0tpXfv3urnlJQUdu/e3WXj6wqECBP/LExBEJAkyWOdOavVypEjR2hubmb48OFERkZSeuaMWkAaPLt5XNeprNeMenMuaAGr5LytN7huE4y0En/a8XhMLhD9cO26tusOHbEwFbRUQuEr0PduCFPiHiQcVuUxHK5GRWxcS5TaPEvl6w3kS/kOR11OT0TnL2HuxxGd6o50lXa0cD1RLTiO0Yb/pO9q0X2Lw+L2oKIkSy67+Du1EAzydLONO6vS3fyjK6m2n7v0/lDo0inIgvpgEiXYmWss4anxsYQbHRJ02jqUdrud2tpaampqOH78OI2NjYSHhxMbG6uKEOj17W/dVVVVLFu2jKSkJLKysggP7yIBZRwRtzNmzOCFF14g2iV9zhPcBVb+2PI/Q4TpJ5TAn7Aw50dlWZYpKyvj+++/p1+/fk4/esFoRGhNNvZkKTn9LTtuMBYr6O0giI6YoXqgohlapMCj/pX+OpP3GKhl6M/8Y2fnAtR2OuJG1MBeCyfXQ/x1EJGGgyzrlcZpT5IKURppI5lAxmADFOvSHbn5SyhVONzEVs0yZT9/K6SU4SDMjt6zqnE81HmArAQg+UIn5+c9tulSxqwtUNVzh47frmc/r/trVyHZtvWSsr2sdO7wBveRGngytoIb+yd4JAudTke3bt3o1q2bo11Zprm5mZqamnZW6IEDB8jIyKChoYH777+fX//618ycObNLichqtTJjxgxuu+02pk+fDjjKfpWVlZGYmEhZWRnx8e1F4VNSUigubhN2LikpISkpqcvG2RUIEaafcEeYjY2N5OfnEx4eztixYzEYDMiyjN1uR5IkwlrTStCkrXh6OpVxKNI0WaDF5nDB6kQQdZBrg3yr48vqKD/4Y2G6I3VXgg/UUvXUZkctXnft+NO/t+WyFRo2AxFgigehOw5CNNDmctW1fg7zc+DuOpZxuHhraS9G7k0dyB1O4LAutbqxSnu+SnAp2x/HfWURf1EN1NBe2b/178bD0HwCwmXn5UGBH65Xp+WC92vHrWWIq0XqvgPZ23atDYcJMpObT/FiskBaXGBCBIIgYDKZMJlM9Orl2FexQjdt2sSGDRs4fvw4F198McePH+fTTz9lwoQJQZu31EKWZW6//XYyMzNZuXKluvzGG2/kzTffZPXq1bz55ptMmzat3b5jxozh2LFjfP/99yQnJ/Puu+/y9ttvB32MXYkQYeKfW0Bb4kuSJL7//nsqKirIyMigW7duTkQJIIoihogI1SXr6f6peF+VYEudDDp72z1apxdoNIRjszUjupCuJ7i6e8E5F97d/u4k+9wRZ2dcoNr2AhFScGelBtKOa3ue1tntIFe3BtFE4lDrCcdBksq9x0D7k+tPB+AgykO0RaRqXbHKPKI/pncjjkCdFjQTuTgbTr6eJBpxCKV3ptDFER/7y46HESGQp7xAA346tK+roo92fhLN3503fUUgWrKxuLaI3wzo6daV2hHodDokSWLv3r2MHz+enJwcysvL+fzzz9m8eTMpKSkMHDgwKH1psWvXLt566y2GDh3KiBEjAHjyySdZvXo1M2fO5NVXXyU1NZX33nsPgJMnT7J48WK2bt2KXq9nw4YNXHXVVdjtdhYtWvSjUxcKEaafUCxMJainV69ejBs3ThU9sNvtyLKMIAgqASuE6c1yUyouaafEFC+g3mRCd8kl1MsyZGUhW61+k6Xr352VolMQrOAhf26inty5Tu34OY6ALHPlKUbJYwy4ATew4pjv04qba61Lf8kSHGkgZ3CeA9WSpj/tnGgdiwH/TqLrNjIOC9NbwJCE7zSkrnDF+ljW/uv0zryuuZrK3+6fmdq+CB2Q3tjEE41WLk/vFVQ36eeff87KlStZu3Yt06dPRxAE+vTpQ58+fZg9e3bQ+nHF5MmTPYq87Nixo92ypKQktm7dqn6+9tprufbaa7tsfF2NEGH6CUEQ+O677wAYMWIEERERyLKMJElIkoQgCO0Cggyt0niuLlkZx724CYexoRClOvVkMGBMTqbplltovuEGEj/+mPKdO9V2A7l3K9t2JuhH219HvZGu8Ec03R8IrRaWJ8vZ03yxX/10liS1KMURkaotzOwqq+fvBPVXOH48CrQn09UocneSbTgCj+x0vIJIPg7C1dF2nlz6stYE2Kbg4d3f/fxZLzg/yLW9nDuV2+2oJc62d+f1ILT+M8kCY8uq+W1YAom9uvt5IL5ht9v5f//v//Hhhx/ywQcfkJaW1uk2Fy1axJYtW4iPj+fAgQMAzJo1iyNHjgCOfM7Y2Fj27dvXbt++fftiNpvR6XTo9Xry8typglw4CBEm3l2ySlDPyZMn6dWrl5r8q3W/aq1KLUS9Hp1mzlMxXKw47lWuwZd6UcQQGUnYlVdy+oYbSLv0UuLj4/nq88+9atI6jdflXft3R75s1z4DmQv1hmARZkeDh9qdH29WZGeJswH4pvXdldgUi9Bf4irE4drVEq/rPKi3tgQc847VBH5cSj8SjmNxox2rRVUWiFaQRTfftee4Gud3X2Px9wfgYw7TU6fe93Eu56X8i7HIzDnZwJIeaV4rjASKM2fOsHTpUgYNGsSOHTswGv0t8Oodv/zlL1m+fDnz589Xl/39739X/161ahUxMTEe9//000/p0SN4mrc/ZIQI0wsaGhrIz88nIiKCfv36qfMPClmKoujTzaI3mRwWJg6SbMZhVeo0Lz1gMBrR9+sH8+YhTJnC6MxMDK2iB3qNa7ej925Jr0cUBbDZHXI33rb1sFxxHfu7r+t4XedU/YG7+VhXBMvq7TIcxOFGVeTjXF/+BvtIOAJ1lKoi7oJ9/CHe73BYqB2931qAo/ic/9Qm7Hv8jvxwofoF13OhfXls17/qJLJmW2+MrpOhX1ULd5XrGJ/YJ6hkuWvXLlatWsXjjz/OtGnTgurevfTSSyksLHS7TpZl/vGPf/C///0vaP39mBEiTNpbmO6CekpLS2lublaVOPwhS3AIsMuCgFUQsMmyak0aaLMqTbGxxN54I7rrr2fQ+PFqOLkCfXi4kyatv3EaMiCLIkRHo7/hBoT9++HgQY/bat9d/1YQiGKQa1vaPgIJp/BGdmIn7hudIlFfJ0pBGQ6SUwTbXcXaBTyTnGu7p1vbc7UutSfTF2HW4VWZxy+U4Ryd64n0tDmYnlzG5xKCV/Z0cct6+ru9S1dAwGQXGVcqscCagGxr5sCBA1itVqKiooiJiSE2NpbIyEiPedyeYLfbee655/jkk0/4z3/+Q58+fQLav7PIyckhISGBAQMGuF0vCAJXXnklgiCwdOlSlixZck7Hd64RIkwXVFZWcvjw4XZBPVFRURQVFVFeXk5MTAzdunUjNjbWp1tk9IoVFGzbxqkvv6SxvBxbfT2SxYJFljEYjfQYPJi4hQtJnDCBtAED3F5QroTpC4rrl7AwwgYNovvvfofp0ksxtQYHePM8up/faYt98RVtC56JUgt/nr399ZAGcgsKRvyOx4a17+Agla90UCvjlMGvfRnwzyq04ZgD1QoVuBKvL6ECcAQL1btZFwiJncSrWAFAU6kjQrZD7QcKX3OfTjypFR7wZd56Du5R1ouIdGsWuP5UFDN6DnG6fmVZpr6+nurqak6cOEF9fT0Gg0FV8ImJiWmX161FRUUFS5YsYejQoezYscPrtl2Fd955hzlz5nhcv2vXLpKSkqioqGDq1KlkZGRw6aWXnsMRnluECLMVVquVw4cPY7FY3Ab1REZGMnbsWOx2OzU1NVRVVVFSUoLFYiE6OprY2FhiY2PbyVgNuP56Blx/PQA1RUVUfPMNpZ99Runnn9Nt3DjirrySIWPHEhUV5XFs2vQUBa4eOaflooihZ0+633orSatWoW9NIlbcw+7gyWDqDDl5a78DanIe3bPnTBA5kLBduw75UCxCRS3Yrc7rtE8d/s5d1uOwVBXrUnHBKu2KeCYw7Vf+PT7JzitqcQgm+HBz1B0BW4Pz4Tn98jwZe50lVg/7Oz/PtPffal2zzr+xNhbWLhMAvSySVKljoWUQgxLaJ+ALgoDZbMZsNquScBaLherqapVEbTYbZrOZmJgYIiMjiY6ORqfTkZ2dza9+9St+97vfccMNN5wXRRybzca//vUvvvrqK4/bKMID8fHx3HzzzezZsydEmBc6ZFnm22+/JTEx0Skx2F1Qj06nIy4ujrg4R1UBSZKoq6ujurqaY8eO0dTUREREhGqBms1mdd+Y1FRiUlPpNn48xmPH6N27N8nJyb7nQcPD1TlMCYex0UjbHKjq5RMERKORmDFjGLBmDTGTJzu1bYiI8H0ufKwTdDrQichWG55LUfhu3x9L1df42lxige3XkfX+Q0DWRyHVXox46FtosDpbgCJtqRjeame64lscX7qyr6s15Q/xluNwp2oZIVCysuBs5XqC3Y+2vLln/eUHf6xKH41786y0/0ocgw636RlRamZuzESiY31fVwrCwsKIj49XlXAkSVKt0C1btvDMM89gMpmoq6vjiSee4PLLLz9v8nGffPIJGRkZpKSkuF3f0NCAJEmYzWYaGhr4+OOPWbt27Tke5blFiDBxEM2oUaOQZdljTqUniKKoulf69OmDLMs0NDRQXV1NUVERdXV1GI1GdQ7j1KlTCILAyJEj/Y5y05tMoNMhCQLNsoyV9nmbelEkKiWFtHnzGHDvvejdWKz6TmhLyoKAEBtL3c9+hvXzz6G8PGDC1G4dzFtAR4Tcva3v+ED0SOY+2HouwvCPvyHU1jo/IUAbcQbiXTsLlNAmg+duis2fwSuu1M7MI37VOg7lzuHBHSoHKogQrB+El3Zkj5VK2nZ0P53QFvAjIBDbEMZlZwZwRc+RnSYzURSJjo4mOjqaqVOn8t5775GZmcmkSZP48ssv+cMf/sCAAQN4/fXXO9WPN8yZM4esrCzOnDlDSkoKjz32GLfffjvvvvtuO3esVoigvLycm2++GXBYo3PnzuXqq6/usnH+EBAiTA38SRXxBUEQiIqKIioqSn0ya2pq4vvvv6eoqAi9Xo/RaKSoqEh14yrRsJ4Q3bs3aVdfTVleHs2nT2NtaECyWBAlCUGWCTeZSLjiCi5+8EF6jBnjsR1DZKRHl6w7qDcMoxExI4OK6dPpMWsWkTNm0FJR4fMe7a6+poJA0krc9aOOTe5c4I/HPgJxvwpAWBT2tOuw9bsDwwurEQqOgmx3zrXUWpr+umIFHDmPdThbl9q+/XlikHAE+1gJXJBYQTNtOVFedBqt9VBX4Hm9T/jzfbojfXef27XljTS9D0gv6+h+Ooab5J+RGh9cDdSsrCxWr17NunXruOaaaxAEQRUg8CQUECy88847bpe/8cYb7ZZphQjS0tLYv39/Vw7tB4cQYeKYV1iwYAGjR49m0qRJDB48OOBoNk9oaGjg8OHDREZGMnnyZPR6PVarVZ3HKCwsRJIkoqOjVTeua5UBc3IyV7/yCrIsU11QQMX+/RTn5lK2Zw9SUxNDFyzg4uXL281zusIQGel1vas8HqKILjoabrqJmltvJWPcOEwmE3ovlrE/aSDQei/zwzLypy1f35QnwnW7rT9EqVkniyJSTD9sl61GSppC2NN3Ix79Fqw2ZzLTlgYLZP7wNMhlIkKL5BxerHXx+qvsU0nnrvjTONJjXNtwcevaW6DpFBi8WZnBdDF4Ik8P7mbnhzF3gT1tOyv/h1vD6VOaxjVxVxJhCJ5Gq81m4+mnn+azzz5j69atbt2fgTy4uxMhePTRR/nTn/5Ez549AYeUnTu1nW3btnHvvfdit9tZvHgxq1ev7uBRXbgIESYO2bvVq1eTnZ3Nc889x6FDh+jduzcTJ05k4sSJjBw5MuAINUmSKCws5PTp02RkZDgl/hoMBnr27Kn+gJVAourqak6ePElLSwtms5nY2Fi6deumBhIJgkC3/v3p1r8/g1qrBDRVVmKK869Kuy/ChDbSFMPDMWVmYp07l+Rp00hKSlIvXDEszGfwkOucULtt9DpEQXAIuErtWckTUQYr6Md13ipgCCAbIqhPvYzjgxZRWxHO4P+7g/Bvc0CytCc2ZaCBkKXOgPU7M7raOgS7pParvivzoP7mXlrxHG3lzz25FM8nS0NQspsvSWj3R4B9dwZu2ndHmu3XCwiIRNWaGVV1GRfHjwrqfOKpU6dYvHgxEyZMYPv27UHRmXUnQgBw//3388ADD3jcz263c/fdd7N9+3ZSUlIYM2YMN954IxdddFGnx3QhIUSYOOYRhg8fzvDhw7nnnnuQJImCggKys7N56623WLlyJbGxsUyYMIFJkyYxduxYIiMjPV481dXVHDlyhPj4eMaMGePTWnUXSKQEAij18CIiIlQXrtlsVtv0lyyhNXhIEBxk58HNI+p0hMXFETltGqZbbuGisWPbzbV6szDdwaknQYDISJomTMBy5AhySYnrFp739bBe8XR2ZG7S7TpfO4g6pG4p2Kbeh37SEjLOlGN44j70eblgtThbfwIaKSc/2gccgUNmKupG070oH7HF5i6w0wFPV7B2+zocka2+omN9cUEJjogzXwStqZju5EEOtlXp7d3bPl5WaiNhdZKeuFO9uUI/kx7xCR0bpxvIssynn37Kr3/9a5555hk1jzEY8CZC4A179uwhPT1dldqbPXs2mzdvDhGmC0KE6QaiKJKenk56ejqLFi1ClmVOnTpFdnY227Zt4/HHH0cQBMaPH69aoXFxcVRVVbF9+3bS09MZOnQoEX5EpXrqXwkESE1NRZZlGhsbqa6upqSkhLq6OgwGg2qBxsTE+KUqEt27N9F9+tB09qxjHtRmQ5Ykh/KPIKALDydm6FBif/lLBl55Jb1aa3u6Qm80+hVn0i6AQqdDN3Ag0StXEjljBpFTptBYWuqXhefNNXvOYggFkMNMSEOuwHrLo8iJFyEc2EfY4w8g5n8FNouz1qHWwjTi+2oTAFFPS0RvDsffyaCvv0FfnQtW2bkt5W9/c3NO4EhLkXFUYekICnDI4WnH6gGlOQ63rBOv+hMNG+gXGcD2nmphulqaQut6fYuJbkcvJvZMOkVhpdTE1KsFnDtjCdpsNp544gny8vLYtm3bOasHuWHDBv7yl78wevRonn322XbiKKWlpWrqCzhqV+7evfucjO3HhBBh+gFBEEhMTGTWrFnMmjULWZapqalh165dZGdns2HDBkpLS7FarUybNo2JEycGtRadIAhERkYSGRlJcnIyAC0tLVRXV1NRUcGxY8cQBEG1QGNjY926kDNmzGDgTTdRefQoFd98Q3FuLqe+/JL68nL0BgNJ06bRe/p0Bo8e7TUQSWc0+gweauc2jY4m+qqrSHzsMQytaiU6H8FO/sDfnE5/3a4e4ytEETmmJ9ZZq7FfOg9Jb0T44wYMf34JTpchS3YE18Aexar0x5svgGwwU959ItUXP8jA5maMe56DZovTNuq9XiFmX2jGUVJMqWxi87KtJ8g4yFIbHesF1kZHvJMKX9afayBTR+GpHy+BP9rPDrrUE1bTi6G1vyA9fRikoxZvPnPmDN99951avFkhUJPJ5JeFqESYXnrppXz00UdBK/XlC3fddRePPPIIgiDwyCOPsGrVKl577TWnbdwFFp2vdJYfMkKE2QEo5HTdddepRVtTUlK49dZbOXDgAPfccw+nTp1i8ODBTJw4kUmTJjFw4MCgBRIBGI1GEhISSEhwuIqsVqs6D3rixAnsdrtTIJFC4KJOR4/MTHpkZnJRK/kf/+orio4fZ/jPf+6XiLIugPQUwWAgcuBAUu+5h/h585wuQp2XuVAF3uYaZQCdiKgTka12t2znLr/O3XqAEjuUNcEAoLeJthJkYeHYh4zHetd6bDHJWDZvwfbKH9B9sxeL3YpOBJ0O9CIIrX8LAgh+6gjKoo6myGQO976F2j5XE28zIr/3AkLVWWf9WQicWEpwWJdKO9U4TL9AnueswNd4FitwHY8cwBC74p7sJdin7eXKriKCZMR4cjijTEuJ7tE21REeHk54eLh6rSnFm2tqajh69Kiae61I4CnVO9R+ZZmHMt5FAAAgAElEQVTt27ezdu1annvuOX7+8593wUF7hjJugDvuuIPrW4VUtEhJSaG4uFj9XFJScs6s3x8TQoTZSRw7dowlS5Zw1VVXATC9NRjHZrOxd+9esrOz+e1vf8uxY8fo378/EyZMYPLkyQwbNiyoT5gGg4EePXqohCdJkkqghw8fprm5maioKNWNGxkZSXNzM4cOHcIUGcllt9zi93gMPqxnGUAQ0EdFkTxjBukPPkhE377tthM7IPXl5EILC6MxNZVaUzgcOgw2/8wnT6RZJcMeC3xmAWocF0ekTqDfgET6VCYjL7gb4fhxdNXV6O12VThfDxgE0Ak4yFMEUzgYfD0fCSDrTZxNnkD5+HsYOGoKdrsd23t/IuyrHFDmLtvf2/2zLq04AnUUsXZwkF6Tm5PgDadxEK4f+1ibQLI6LxNcP/iaZ/SHRH1t49ezqWJZiggtMcSWzWBYz5l+xRx069ZNdWsqUyY1NTWcPHmS2tpaZFnmr3/9K2PGjOHgwYMcPXqUjz/+WBVGOZcoKysjsXV65YMPPmDIkCHtthkzZgzHjh3j+++/Jzk5mXfffZe33377XA/1B48QYXYSl112mdvler2eMWPGMGbMGFatWoUkSRw5coSdO3fyyiuv8O233xIfH8/EiROZMGECY8aMCaobVxTFdhe1EkhUUFBAdXU1NpuNxMREEhMTA7J+w6KjCYuObpsHtTtbd6LBgLl/fy5etYp+c+d6bEdnMPg1F6qFTGsCemwsDVOnIi1eTNyf/8zZw0c6JTzgTrjGBtTYZfYd/p59h78nAugOxAFmHEZaGK2EKUOYDOESROi9uHbVuUgdNnNPCoZNJ+LG1aR1d9RM1JcVYd7xAWJtg4OktGkj2vlLf9BkghMStFgc5rKiMqSU5lLG4wsFOEjWD8dCbQnUlYLR1crswvlJj9uKLusFZzECAEEOw17Znz7NK0lKaE8kfnWvmTJRrLKmpiYOHTrE3/72N8rLy4mKiuLBBx9k4sSJzJ8/n0g/ItY7AnciBFlZWezbtw9BEOjbty9//OMfAWcRAr1ez4YNG7jqqquw2+0sWrSIwYMHd8kYf8wQfCTFdm3G7E8YsixTXFxMdnY2OTk55OXlYTKZ1ECi8ePHExMTE/R5hIaGBg4dOoTZbCY5OZna2lqqqqqoq6tDr9c7BRJ5sjjtFgtnDx+mfN8+inNyOLV3L01nzmCtr0dvMDB4xgxGP/QQZk0QgTtsmjOHgg8/RLJa260TcdyfI3EQU1Tr56iwMCL796dx8WL63XwzPRIS2HbXXeS/847bdoTW/SJaX0p7kZrPSj8n9JBlaxPUcUWYpp2o1neTph3lc6QeIk0QFt7auKn1pfwdHU512nCKpyynzyU3OM0XG9Y/jP6fr0JTQ9sBaF+61naMmg61/UQAJhE5vjfyVz0Q9x0Ea7NzWyIO1s9sPRDlwJR2tagFtgBVre0bNX1GtH6ObPt8phCObwKjRXOu9BAVAQZv+2qXK+fLmxWttBWh2U+7v/bcmMBugqYIkRaDkSbBRAsmmmw9aDw5hf7m+4kI9z/a3BdkWWbbtm089thjvPDCC1xxxRXY7XYOHDjArl27WLhwYVAfjkPoEri98YYszPMEQRBITU1l3rx5zJs3D1mWOXv2LLm5uezcuZNnnnkGm82miilMnDiRhISEDhOoJElqtRVtXmhUVJT6VGyxWKiqqlKDGwB1XqZbt25qIJEuLIz4YcOIHzaMofPnOwkqCHo96ddei+iHe1fnwyXr9LQmiuhjY5F/9jNYsoSLx49XCV3sQPCQW+EevR5kCeyeM+69zYn6zOsUBCRzHAVjb0J/8wrS+6Y7rRZ3Z6P7cDPUNbaZvK5WpTdHgACSGEZVtxHUdLuF3iUbES2WNoUgtSMc85r+oJGAik3LLumi7sboVwpIsKCdwxQcnQvosTUmQ8VShsTPCupDqcVi4dFHH+Xw4cNs375dnT/U6XRq6loIP16ECPMHAkEQ6NGjBzfddBM33XQTAHV1dXzxxRdkZ2fz6quvUlVVxfDhw9V80H79+vnlSq2rq+PQoUN0797da15oWFiYUyCRzWZTK7MUFxdjs9mcKrMo0YFaQYVAoPMjn1PGQWSmwYPh1lvpP28ePVoFH9R2fBCmr7RKWRCwR0ZQP2ok8pdfQWOjlz0C7wMAfRgNqQM4On0VfafcQLhrylFDPYbf/38IpWVg0/gzlcAhX0WmBQE5qif2y+7EOHkFPV58Et2pk2B1I3gg48jJdIWi7K91A+8CWvBbyq/iGE46sn5RUUciZP1JU3G3UjZSVzGKSOkxeiQE1+VYVFTE4sWLueaaa3j22WeDUkDanXLPgw8+yH/+8x/CwsLo378/r7/+OrGxse327du3rxqApNfrycvL6/R4fuoIEeYPGGazmalTpzJ16lTA8fT65ZdfkpOTw+rVqzlx4gQZGRlqJO5FF13kdJHa7XYKCwuprKzkoosu8lpCzB30ej3du3ene+v8miRJ1NbWqsIMzc3NREZGqpG4UVFRAT2tG8LDEfV6ZEly5IO6TA8IgkBYt26YL7uMuLvu4qKxY92mu4gGg89oWy2cejEaMQwaROTjj2MqL4e8rwNvwxtEASmmG8Xjp2BdsIpBGUPanyOrFd1LzyAc+NYx3whtN36tfqw7SToB0IdhTx2ObcbjSAMuRVdRRkT2dsSGZuf9vc2BatNOFMKsbX25M7g9pG5UFYFgcxWr8GN/7fEEAn/yOFuXS7YYKkpuIzHu1xjDzAF25BmyLPPf//6X3/3ud7z00kse4xo6AnfKPVOnTuWpp55Cr9fz0EMP8dRTT/H000+73f/TTz/1K/I9BP8QIswfEcLCwpg0aRKTJk1i9erV6rxIdnY2zz77LIcOHSIlJYVJkyYRExPDxo0b+fOf/8yoUaOCktIiiqJqXfbt21etzFJVVUVhYSH19fWEh4erLtzo6Giv/V581130HDaM4pwcKvbvp6myUg0kEgWB6EGDiJs+ndG33+71ovfl2gU37lJBQIiOxjxnDj0eeghdfDwRmzb51OP1GwKg19OUNpDjN/+CxNm3ewz0kD75COGtN7DVN7WWaVOH6MjtdEeWrZ3IpmjsV8zHeuNDEOkI8NL9bxvid8fBorEutRam1vCRcQQXNeOsTKTDER17unW534msHpYHI4gnULS2IyNS19CXE0W/ITnpOsIMwQu4aWlpYe3atRQUFLBjxw5V7jJYcKfcc+WVV6p/jx8/nvfffz+ofYbgGSHC/BFDOy+iSPodPHiQBx54gPz8fJKTk3nooYcYP348kydP9inpFyi0lVkUlZCmpiZVE/fw4cNqIJHy0gYSKfmgIxYvRrLbVUGFE9nZ1NbXM2DxYoaOH++zmkug6SmiwUB4//70eeYZzJdcgtDavqjTBedeLQhIUVGUXXY5NcseIG3ESI8PDvb9+7E/9SS2ikoEO4itaSmC4KjCIoqg07vhEL0eObEPliXPIg27AsRWFpQkDBvWOwQPtJGw7ghToo0oFZev0PbZvltEtMkIYZ6EYV2OxUY7SbwOwdeO2iAo3LxrIGGivGoSTWG/I8xg5tix4zQ1NWEymdTfpK8HO08oLCxk8eLFTJs2jRdffDGoedb+4rXXXmPWrFlu1wmCoMruLV26lCVLlpzj0V14CBHmBQS73c6SJUuYP38+H374IYIguJX0GzdunBpIFBcXF9SgB5PJhMlkUvO+lArzlZWVFBQUIMsyMTExqhtX0alVBBXscXHUpKYyOj3d76d1vxWDBAFDdDQJ06aRvnYtRhfpP0EU/XLtehM/kHU6WlKSKZ47h4S776WPRnTfFfbiYhofegi+zUe0SSpJWnEQpdCa12nSabhAAEyR2Cddg3Xpk8g9kp3aFF/9A1LFaQSrvY1H3AUN2XFUL4nGueqJCBhEWuw9sLfYMcmVPs+Hgu+/gpYGv7JP3CMY7liVTEWsUhyldb+ke9/fEKv5jciy3O7BTqfTqco9npSytPv/+9//Zt26dbz88stMnjw5wIEHB0888QR6vZ7bbrvN7fpdu3aRlJRERUUFU6dOJSMjg0svvfQcj/LCQogwLyAYDAZ27NjhpGHrS9KvoaGBkSNHqoIKKSkpQSVQ1wrzSmWWqqoqSkpKsFqtmM1moqKiOHv2LAaDgdE+pPlcIRoMPkXlBZ0Oc0YGQ1evJvGGG9xG1oqKRE8HIURGUjtuHLV33IHQpw/5+fmEh4erDwdaS8Z++jTVDz6INS8P0WJVDT+Vt1rJUtTet0UROTEV68JV2K+5DcKcg6bkinJsf38Xub5ZTbtUTotyWGrtUDuO+pipOAJ6WsUQZKORqvjhNDKJhOpXENxUknEcrMs7jmAfWW632O1n3yv8hFvr0kCNbQA1kb8lvt9V7X7PgiAQERFBRESEGiGulNyrqamhqKgIm82G2WxWCTQiIgJRFGlpaWHNmjUUFxezY8eO8zY/+Oabb7JlyxZ27Njh8XpVji0+Pp6bb76ZPXv2hAizkwgR5gUGb4LvWkm/6667DnC4UHfv3k12dvY5kfRzV5mlqKiIwsJCwsPDaWlpIT8/XyUZs9nsk8B7jRxJv6uu4vSBA7RUV2NtbFQFFQTAEBVFn2uuYdxvf0ukm3qD6vkRxYDu3wqVCDoddO/OmYUL6X3HHaS1Wq6yLNPc3KxaMkeOHEGn0xETFkbYE08gb9+O0NLSTn5WAEQZTFqXapgR+8TLsd7/W+R099Gd9jffQDp6DHuL47gVohRbXzqdS1aKBUdQTxQgCtgjYyjMuJmomx/GfNttYLX6TGNxOh/eynoFCn/csm6WSUIkJ20/Jzz9aXrGeM8D1sK15J4kSdTV1VFdXc13333HHXfcQVRUFFVVVUyZMoV33323y8QHfGHbtm08/fTT7Ny50+P13tDQgCRJmM1mGhoa+Pjjj1m7du05HumFh5BwQQhOsNls7Nu3j507d5Kbm+sk6Tdp0iSGDRsWkPXnDRaLhSNHjgAwaNAgwsLCVJmxqqoqqqurqaurw2g0OgUSeQrXt1ssnDl0yFFgOyeH8r17EUSR0XfdxeAFC3wG9BTu2MF/5s+npbra7fow2gsVRJhMRAwfjmHlSjKuvNLnuWkoLKT4gQewfvIJotWKnvYeUYHWnH4BTBEiEck9sC9YjG35Q+Ahv1U6epTmXy6A/IOIrcrnWutSEMAY1mqUuogEyLF6Gnv1peim35By6TQ4cABp7kwi6k5hMMjexQZaBRRaJPj6E2goBpPs4GBFcMJkAHME6BWxBaWtMNqUJFyFC8LxTNaCZpvItpPVHNGL0m7LSRhyJzpDhx3D7SDLMv/617/YsGED11xzDadPnyYvLw+9Xs+bb75Jenq670Y6CK1yT0JCAo899hhPPfUULS0tavT6+PHjeeWVV5yUewoKCrj55psBxzU9d+5c1qxZ02XjvADh9pEtRJgheIUi6acoEn3zzTckJCR0WtKvoqKC7777jrS0NCdxaHdQrLSqqipqa2udonVjY2M9kpQsSTSeOUNkqzvYF05kZfHvefMchOnmulA0yyOBSFEkumdPDNOmkX733ST6cdOsysmhaO1amvfuRbRa2xGlEndjpFXVKNyAaeRw9I8+jjzJe6pC3f33Y/vbXxEbGtu5YgXBcfVHRrgQZiRIsWbKhk/G/svH6DlgMLIs07J+PTzzBFE6q291HqPjxDQ0Qs4W0DW1nh/8IEytWlFHCLP16UU2GaiMGkrzRU8T12eiz+8hEDQ1NfHwww9z+vRpXn31VdUzAlBbW0t4eHjAxeVD+FEgRJidwbZt27j33nux2+0sXryY1atXn+8hnRe4k/QLDw9nwoQJfkn6Wa1Wjhw5giRJZGRkdOhmo8w3KS9Jkpwqs4QHUE1Fi+KcHDbNmeOTMM1hYcQOHkz3e+5h+E03tSuw7QpbQwMFGzZQ+uc/YysrQ5RlVT9dmbdUhNzDgChBwBgZif26azk5azaG+HiniE5XC7slJ4fqJUugpBhBkp3cu2ocj6FVrk8hqQiRltQUCq+aR/wv7leFFOwnT1Jz5ZWYTp7AbJS8E6ZS41OE+gbI3Q66lvaEGWFwSOPpXQkyCIRpN5s5GXc9kZc+iynSc4BVR6AUVpgzZw4rVqzo9NSEOxGCyspKZs2aRWFhIX379uUf//hHu1qVELr/nAeECLOjsNvtDBw4kO3bt5OSksKYMWN45513QtXIwUnSLzs7my+++AKr1epW0m/btm1ER0eTlpYW1KoNSiCRQqAtLS2YzWbVjRsREeFXINOpr79my8KFNJ09i62xEcludxKWDxMEYnr0oPuUKQx54AH6ZGR4bdfW1ETJ9u0c+v3vqTtwANFqVYlRW2daWWYCIvR6YlJT6fn732O88koEnU6N6Kyurm5nYcdERFA1fTqWPXsQLBanYFhtWmWkoU3fVo41cnbYSKrvWE3SpClOx1D/yis0Pv44kY01xJha9V+11qBCmAbnAzh5Cr7ZCzqLY7Wq/UsAhGlyWedJKEcAInU0mXtRftEqeo69I6jz7LIs8/777/P888+zceNGxo4dG5R2s7OziYqKYv78+Sph/upXvyIuLo7Vq1ezbt06qqqq2okQhO4/5wVuL+xQ0I8f2LNnD+np6aSlpQEwe/ZsNm/eHPrBgl+SfqdPn8ZgMBAWFsbzzz+vRswGC+4CiZTKLMePH6exsZGIiAiVZMxms9sbbK+RI/nl7t2cPniQ8r172wQVqqqwNTQQM3AgSbfdxqSFC70GVwHs+9OfOPjaa9QeOQItLejBqRyYligVY0qMjMR81VX0evJJDBrhetdUHcXCrqqs5OQTTxCRl4eulSzdFTZReUcUscfEcOLa6USuepTkOGfBcXtlJc1ZWVjr67HgRa2nhbZi0q3MfKIQLBb3KSUBB/4oEn3aai2aaFhZZ+RM9MXYrniOhD7B1WZtamrioYceorq6mk8//dSttddRuBMh2Lx5M1lZWQAsWLCAyy+/vB1hhu4/PxyECNMPlJaWqon54Ci2unv37vM4oh82tJJ+H330Eb/61a+YNm0aRqORJ598khMnTjBw4EBVtchV0q+zEEWR6OhooqOjSU1NVQOJqqurKSkpoa6uDoPB4FSZRelfHx5O4qhRJI4apQoqlOzbx/6PPmLA1Vcz6OKL/bJWv3njDU4fPKhG6ipkprhhFaIMB8JFEWNKCim/+hXxs2cj+pgTViI6TUePUvTvf9PS0uKUkiJo3pULXAgLo6F/GhV330/yLXPcnm/r4cM07dqFbLfTJLgQphIBa6WtGLXGhJVbS7y4VarzR75OgaI6ZAF3fmuLMZqCpGk0DrudbsYEmpubO+yCd8WRI0dYsmQJCxYsYNmyZedEiKC8vFx9EEpMTKSioqLdNqH7zw8HIcL0A+7c1sEuu3UhwmazsWXLFrZt26beFACPkn5KKsvFF1/sc14wEGjrFSYnOxL9W1paqK6upqKigmPHjqkpN8pLidgtLinhlM3G5ffcg9nsv/6oNiJX4Rqttrmx9aULDyduwgQG//73RAVgMbScPEnJb39L89mzCLKscpiWnA2AXhCwm0yUXn4ZzfesIHHIUI9tVq5bh7WuzqHPLoNNaq32pcjnNdKexESwCgJ2nQEEq5dCoG6gvYQUQXgXy1XtRy9S3y2NM5c+TPLYGdTW1akFm1taWtTi6B3RNJZlmXfffZcNGzaoUpI/JITuPz8chAjTD6SkpFBcXKx+LikpUZOCQ/AMvV7PSy+91G65O0m/goICsrOzeeutt1i1ahUxMTFdJukHYDQanSqzWK1WdR70xIkTWK1WbDYbUVFRDBkyJOCcO9FgQNTrkaCdsLyM44YXnpDAgF/+kotWrkQfQPuWigqO3n8/tV9+iShJCILgCCQSBITWgCIBx3mWEhM5feedpMybR5PdTnl5OUePHkUURTUpPzY2lpacHOrz8x06vjg4qsYKJiOILbQxvXbiVQDCwyjvnkZFUwl62V0JFA/QfpUSDrJUWN9Kmy9ZBDncSHnyJJj1LAm9BwKoLvh+/fo5FUc/ceKEUyqSot7jyYPR2NjIAw88QFNTE1lZWWrZu3OFhIQEysrKSExMpKyszO10Rej+88NBKOjHD9hsNgYOHMiOHTtITk5mzJgxvP3226GK5F0EWZZVSb+cnBz27NmjSvpNnDiRiRMn0r179y55ypZlmbKyMgoLC0lJScFut1NdXU1zc7NqxXTr1s0ngZfl5XHyyy8pzs5uJ6igEwRSRo5kwm9+Q/IVVwQ0PltdHYcffpgzmzYhNTU5CFKWEVpdvzpAlGXCw8LQjRhB9OOPkzpxYruxah8QqqurER55BMMXX6iBQ4o716QHsx66h4NR6/fVgxxtxn7rfIoGjuPru+7GUFen5qhq8zAjwhxBPzol71IbcRtGGwlr/datEn3WqFhKx/2CuNseCyj/V0lFUgKlwFHbNSYmBovFQmpqKocOHWLJkiUsXryYpUuXnhMXbGFhIddff71Tqa7u3burQT+VlZX8/ve/d9ondP85LwhFyXYGW7du5b777sNut7No0aJQEvA5hCLp99lnn7Fz504+++yzLpH0s1gsHDp0CL1ez6BBg5yE4rVWTFVVFQ0NDX4LeNstFs4ePkz5/v2U5OYSFhXFxF//GlNr4rm/sDU28s2aNZzeuhW9Tofc1AQWi6Mmqc0GNhuCJGGMjUV/440MevhhYnzkuALU7NhB4R13YK+oQJBlJ9eu8ooRIMkERj2IBhEhtQ/WR3+PdNlUijZv5tu778ZQX+9EmBGt7yZXwlSiYBXXq1YXUHH3hhmoTUijevZj9Jh8fae/W6W2a1FREffddx9nzpyhoaGBO+64g7lz55KZmdnlhOlOhOCmm25i5syZFBUVkZqaynvvvUdcXJyTCAGE7j/nASHCDOHCgVbSLzc3l7KyMoYMGaJaoIMGDQroBnj69GmOHz9O//79/Yri1Qp4K1aMEkikuAH1HlR5OoK64mIOPPMMtQcPIggCcmMjssWCThSRqqsdlVYsFsSEBOKWLWPoHPeBPa6Qmps5vGABtR99hGi1uiVLHQ6eMwsQFWFEuGQSxvUvIPZzFAz/cOxYrIcPY7LbVWEHJ8I0QpRJQ5gG2ufWaIhTMkVQNuIKwu5+mqikvkE7hwD19fWsWrUKm83GsmXL2L9/P7m5ueTn5/Ppp5+q6jkh/OQRIswLBcXFxcyfP59Tp04hiiJLlizh3nvvPd/DOq9QJP0UN66/kn42m42jR49isVi46KKLOqXaYrFYVEm/mpoaAKfKLB1tu3D7do7+/e/Y6uoQZdlhSba6du0NDYiArbkZoW9fhj/2GIn9+/vd9sk336Tg17/GXl3tFDDkKtMXJQiYevbEMG8elTOmU2NxhMXGxsZyeOZM5OPHiZBllTDNaFyyRkcOqC4MZxdsO3UFgZYeCZTefCfxc5ejCw9cQcob8vPzWbp0KUuXLmXx4sVOD1SyLIcCaULQIkSYFwrKysooKytj5MiR1NXVMWrUKDZt2hTKy9JAK+mXm5vL/v3720n65ebm8uWXX7JgwQKSkpKCfsNU3IAKidpsNqKjo9V50PDwcJ99njl0iH0bN9JQXg6t85SiILSlJkoSzRYLMRMnMuneewOKLm4sLGTf3Lk0HjzoaFsTMKS8DECkwUC3YcPo8eijmDRzrjabjZPffEPe3LnoSkqIkGWHiBAawhQcc5gR4Y6SnU5FsdUIWJCNBmrSM6n71Xp6jJnk9zH4A1mWeeutt/jTn/7E66+/zrBhw4LaPjhSUrR1KQsKCnj88ce577771GVZWVlMmzaNfv36ATB9+vSgCqKXlJSwa9cuj/UxQwgIIeGCCwWJiYlqmobZbCYzM5PS0tIQYWogiiKZmZlkZmaydOlSJ0m/f/7znyxsFR+4+uqr+fbbb4mMjPQq6dcR6PV6unfvrrr5JEmitraW6upqDh8+THNzM5GRkaoF6i4dokdmJj9//nlszc2qoMKZ/HwaKyqwNjVhj45m6IoVZEyaFNDYLVVVHPzNb6grLASjEdlqdQQOCQKCJKnBRMbISLrNnEniI4+gd5kP1ev1NOblYa2oQN/64N1uBDLY7NDQ5KiWohNbq6YIrQQqgmSM5uTkK4lc+ww9egZX1KKuro77778fnU7Hzp07iYqKCmr7CgYNGsS+ffsAR9pUcnKyKn6uxSWXXMKWLVu6ZAw7duwgPz8/RJhdiBBh/shRWFjI3r17GTdu3Pkeyg8agiCQmprKqFGj2LBhAytXrmT+/Pl8/vnnZGdns379eiwWC6NHj2by5MlOkn7BglbSrm/fvsiyTENDA1VVVRQWFlJfX094eLhTZRbFbagVVJBlmaITJ/hu924uvuIKuvlZaFuL46+/Tk1pKVJCAjpAqq1Fam5GsNsRJQnsdow9etDnoYfotWiRx3ZcU2ac1uHQILDaHDcanVJmTHQUyNaF6bClJFF1xzLilywPetDNt99+y7Jly7j77rtZuHDhOXO57tixg/79+9OnT59z0h9Abm4uK1euJDY2lo8++ogPPvhAtWRDCB5CLtkfMerr67nssstYs2YN06dPP9/D+VHgwIEDyLLM0KHtE/i1kn67du2isrKSYcOGqYIK/fr16/JISiWQSKnMotfrnQQVJEni4MGDmEwmBgwY0CGFpMItW2g4cQJrZSVNp04hNTRgb2zEWlWF5cwZdKKIOSWFwb/7HdFuzpMCS20t/5s3j+qsLCI0AT/KuxLw6i4IVjSG0TB8OOVL78R88cXq8QVDsEKSJN544w31NWTIkE63GQgWLVrEyJEjWb58udPyrKwsZsyYQUpKCklJSaxfvz6oqSFXX30169evP+fHe4EiNId5IcFqtXL99ddz1VVXsXLlyvM9nAsSFouFL7/8kpycHHJzc1VJv4kTJzJ58uSgS/p5GoMSiXvmzBmampqIi79kN7IAABSOSURBVIsjKSkpYIKRJIm6oiJsTU1YampoqapC1Oup//57pJYWpKYmmkpLierblwGLFhHmI4m/6fRpPpg4EU6eVPMulepdWqEerTCQXhAQoqJonjmT1McfxxAV5ZQParVaVeH82NhYv4XzFdTW1rJixQoiIiJ4+eWXz3mRZ4vFQlJSEgcPHmxXtk4Rzo+KimLr1q3ce++9HDt2LGh99+vXj6NHjwatXu1PHCHCvFAgyzILFiwgLi6OF1544XwP5ycDu93OwYMH2blzJzk5OV0u6aft99ixYzQ1NZGRkUFTU5MaSBQIwdhaWrC3tCDZbA4JPL0eS20tsiQhW600njpFbHo65n79EP14ECjJymLnwoUIFRVqdRJtNRZFcF4lTr0eISkJ029+Q/+5c92OU5Ik6urqVAL1VzgfYP/+/Sxbtoz77ruP+fPnn5eo182bN/Pyyy/z8ccf+9y2b9++5OXl0aNHj073e/bsWaZMmaLOo4bQaYQI80JBbm4ul1xyCUOHDlVvHk8++STXXnvteR7ZTwuSJPH999+rBLpv3z5V0m/SpEmMGzeu05J+dXV15Ofnk5SU5FacQUswVVVVNDU1ERERoQYSmc1mj/1LNhvWxkZkWcZSV0dEz57oAyD8rHvv5bvXX8dktxOOM0G6FsXWh4ejGzuWtPXriQ/ADanNd62qqqKurk51U0uSRM+ePYmJieHVV1/lb3/7G2+88cZ5DX6bPXs2V111FQsXLmy37tSpU+q8+J49e7jllls4ceJEUIg9Ly+PtWvXqkIHIXQaIcIMoetgt9sZPXo0ycnJXRYF+EOHLMuUl5eTnZ1NdnZ2pyT9lKjesrIyBg8e7Hd0p1KZRbFAtbqqSiBRMNzILbW1ZK1YQckHH2DQ6M/qXF+CQFj37kTMns3Ihx/GFBvb6b4VN/WmTZvYuHEj9fX1mM1mHn74YaZMmeIk9H8u0djYSO/evSkoKFA1aV955RUA7rzzTjZs2MAf/vAH9Ho9JpOJ5557jokTJwal7/r6eq644goaGxvZuHFj0Nr9CSNEmCF0HZ577jny8vKora39yRKmK1wl/Xbt2kVjY6NPSb+Wlhby8/OJiIggPT290wSn6KoqgUTaaN3Y2NgOzXmV793Lf2bNwl5V5ZDTs9kQJckhAi9JDtesKGIaOJABa9eSeeONQXeRfv311yxfvpzly5fTr18/du3aRW5uLqmpqfz5z38Oal8h/OQQIswQugYlJSUsWLCANWvW8Nxzz4UI0wsUSb+cnBxycnI4deoUgwcPVi3Q/fv38/nnn/PII48EZW7LHZQC1MpLkiSnyiX+1Je0W62c/vZbKvbtozQ3l9P79tFy9qxDEF6S0On1xEyYwOXPP0+Pvn2DOn5Jkti4cSN///vfefPNN8nIyAhq+yGEQIgwQ+gq3HLLLTz88MPU1dWxfv36EGEGAEXSb8eOHWzcuFFNefEl6RdM2O12p0jVlpYWNZCoW7duPiNVZVlGttupOnaM8v37yf/vf4nq358pa9YEfezV1dXcfffdJCQk8Pzzz2PyUWy7I+jbty9msxmdToderycvL89pvSzL3HvvvWzdupWIiAjeeOMNRo4cGfRxhHBeEVL6CSH42LJlC/Hx8YwaNYqsrKzzPZwfHZQAln/+8588+OCD3HHHHRw9epTs7Gw2btzI/v37iY+PZ9KkSaqkX7BJQqfTqfUlwWHBKZVZjh8/7jNSVRAEBL2esJQUamtqmLBuXZdIDebl5bFixQpWr17NrFmzujQK9tNPP/Vo4X/44YccO3aMY8eOsXv3bu666y52797dZWMJ4YeDkIUZQqfw8MMP89Zbb6HX62lubqa2tpbp06fz17/+9XwP7UeDqqoqTp06RWZmZrt1Wkm/3Nxc8vLyMBqNaiTu+PHjgy7p524MjY2NqgVaV1enVmZRAokUfePBgwcHPfdRkiT+7//+jw8++IA333yTgQMHBrV9V/hK91i6dCmXX345c+bMARyyeFlZWect2CiELkHIJRtC1yIrKyvkku1iyLLM2bNnyc3NJTs7my+++AKr1cqoUaO6TNLPHVpaWqiurubs2bOUl5cjiiK9evXqdGUWV1RWVrJs2TJSUlJ47rnn/Jpf7Sz69etHt27dEASBpUuXsmTJEqf1119/PatXr2by5MkATJkyhaeffprRo0d3+dhCOGcIuWRDCOHHDkEQ6NGjBzfddBM33XQT4Czp99prr50TST+j0YjRaKS2tpaLLrqIuLg4dR60qKhIrcyiEGhH3Mi7d+/mvvvuY82aNdx6663nTIhg165dJCUlUVFRwdSpU8nIyODSSy9V17szMkKlwX4aCFmYIYRwgcFisZCXl6e6cQsLCxk0aJBKoIMHD+5UqoosyxQWFnLmzBmGDBnilgwlSXIKJGpubiYqKkolUG+CDpIk8dJLL/Gf//yHv/zlL6Snp3d4rJ3Fo48+SlRUFA888IC6LOSS/Ukg5JIN4aeD6upqFi9ezIEDBxAEgddee40JEyac72GdF3iT9Js4cSIjR470W9KvpaWFgwcPEh0dTVpamt+WqyzLaiBRVVUVDQ0NmEwmNZDIZDJhNBo5e/Ysd955J/379+eZZ57pEqlBb2hoaECSJMxmMw0NDUydOpW1a9dy9dVXq9v897//ZcOGDWzdupXdu3ezYsUK9uzZc07HGUKXI0SYIfx0sGDBAi655BIWL16MxWKhsbGR2CCozFwIUCT9FEUifyX9zp49y9GjRxk4cKBa47Oj0EreFRcXs2TJEiIiIqisrGTRokU88MADmM3mTvXRERQUFKh1LG02G3PnzmXNmjVOij2yLLN8+XK2bdtGREQEr7/+emj+8sJDiDBDCAxff/01p06d+tFp1NbW1jJ8+HAKCgpCc0t+QCvpl5OTw+7du50k/UaPHs0TTzzBpEmTmDFjRtCtPrvdzosvvsgnn3zCrFmzOHr0KJ9//jmCIPDuu++e07qSIYTQilDQTwiBISEhQdXElCRJXS4Iwg+aiAoKCujZsycLFy5k//79jBo1ihdffPGcl3r6sUAQBHr16sXMmTOZOXOmk6Tfv//9b5YtW0ZGRgaiKCIIgkdJv47g9OnT3HnnnWRmZvLxxx87RdfW1dV1iTBBCCF0FF1bDTeEHzWef/55+vbti8ViQRRF9aW9UWqJ9IcCm83G119/zV133cXevXuJjIxk3bp153tYPxoIgkBsbCw2m43du3ezdetW/ve///GLX/yCkydPsmLFCsaPH8/tt9/Oq6++yqFDhzr0O8jNzeXGG2/kzjvv5Nlnn22XimI2m9HrO/5MX1xczM9+9jMyMzMZPHgwL774YrttsrKyiImJYcSIEYwYMYLHH3+8w/2FcOEj5JINwS3q6+vp1asX9fX1/PWvf+Xtt98mMzOTXr16MWfOHFJSUtzuJ0lS0FMYAsWpU6cYP348hYWFAOTk5LBu3Tr++9//ntdx/djw9ddfk5aW5nbuV5H0UyJxjx49SlpamhqJ603Sz2638+yzz/K///2Pt956q8tcroqYwsiRI6mrq2PUqFFs2rTJqfxXKHc4BA8IuWRD8B/ffPONGvzw2WefcfbsWWbPns1TTz3FyZMnef7553nvvfeIjY1l6tSp6n5aspRlGUmSEAThnJJor1696N27N0eOHGHQoEHs2LHjvNZI/LHCmz6qXq9n9OjRjB49mpUrVyJJEkeOHFEl/b755ht69uypRuIqkn7l5eUsXbqU4cOH88knnwRN4MAdEhMT1VQPs9lMZmYmpaWlod9CCB1GiDBDcIvNmzdzzTXXqPNZK1asYMyYMcyYMYNt27YBsG/fPqqqqpg6dSr19fVs2rSJ8PBwbrnlFmw2G3q9vl2+n91ub+fW7Qq89NJL3HbbbVgsFtLS0nj99de7tL+fOkRRJDMzk8zMTJYuXeok6bdp0ybWrFmDJElUVlby8ssvc911153TefDCwkL27t3LuHHj2q37/PPPGT58OElJSaxfv57BARS4DuGnhRBhhuAWH3zwAQ888ADHjx+npqZGLUhbWlqqankKgkDf1tJN77//Pl988QVz586luLiY559/np07d3LZZZexfPly0tLSANwmzNvtdpqamvwukuwPRowY0a7KRAjnDoIgkJqayrx585g3bx6y/P+3d78xNf5vAMffh04UlVgq/RMRy2nGDtaaRVbfkpgSDTHGmDSzPPCE9cTMgwhjYx7EbM2/aSNNf+R/Qk3F5IxCdZIzNX9COef+PejXzVE/+oVOx/d6Par7vs/OddrZfXVf9/W5boX6+nqMRiMzZszo11jev39PQkIC+/btw9XV1WrftGnTeP78OcOHDycvL49FixZhMBj6NT5hP6TpR3Tz+fNntdP0wYMHODg4EBgYCEBjYyNjxowBOk82LS0tKIpCVlYWS5YsQafTkZKSwrx587h//z5ubm5cv34dgB07drBnzx6Kiop48uSJ+n4NDQ1cvHiRp0+f9v+H7Ud79+4lJCSEKVOmkJyczKdPn2wdUr/RaDT4+fn1e7Ls6OggISGB5cuXs3jx4m77XV1d1X/UYmNj6ejowGQy9WuMwn5IwhTdODo6UllZCUBbW5s6mqympobXr1/j6ekJwMSJEyksLOTs2bMEBgYSGRlJYWEhRqOR7du3k5SUREVFhdpQUVRUxN27dykoKGDOnDmcP38egKNHj3Lu3DnMZjNgPavTYrH0OLvT3jQ0NLB//37u3btHdXU1ZrOZnJwcW4f1V1MUhbVr1zJ58mS2bt3a4zFNTU3q96usrAyLxfLLQxnE30tKsqIbjUajNkakpKSo27VaLWFhYfj5+QGdAwIqKyspKChg8+bN6ra5c+dy6NAhSkpKKC8vx8XFhfb2doxGI/v27UOv1zN9+nQuX75MVFQUFRUVVFVVERcXR2ZmJnFxcdTV1eHn56eWcBVFQaPRdD6s+L8/D+S1oD358uULHz9+RKvV0tbWpl6piz/j5s2bnDhxAp1Ox9SpUwHYtWsXL168ADqn9pw5c4bDhw/j4OCAk5MTOTk5dve9Ev1HEqb4oW+XiYwbN45Nmzap+wIDA0lKSiIkJIQ5c+YAMGzYMJqamgCIiIggIiICgPz8fLy9vdHr9bS3t6MoCq2trTg7OzNr1iwSExNJTk7m7du3ZGdnc/z4cUwmEzqdjiNHjuDs7ExHRwdardYuT2g+Pj6kp6fj7++Pk5MTUVFRREVF2Tqsv1p4ePhPqxOpqamkpqb2U0TC3klJVvzQ98tEvj0BeXt7c+LECdLS0tRtS5YswcfHh9DQUGJiYtQZnFevXlU7FBVFobKyEp1Oh8lkorGxERcXF4YMGUJNTQ1ZWVkcOXKE0tJS3NzcuHz5Mh8/fiQlJYW9e/eyc+dOSktLefPmjVVsA1lLSwu5ubnU1tbS2NjIhw8f5CHbQtgZSZii174vg1osFvW+Y5fBgwdz4MABrly5wrp165gwYQLQmTD1ej3QOfLs0aNHhIeHU1NTg4ODA+PHjwc6u23DwsIYP348Tk5O+Pj4UFBQQFtbG/X19Tx+/JjQ0FDOnj1LTEyM+r5msxmj0fin/wR9VlhYSGBgIB4eHmi1WhYvXsytW7dsHZZdyc/PJzg4mKCgoB4nNymKQlpaGkFBQYSGhlJeXm6DKMXfTEqyos/+1zACRVEYNWqUVVdicXGxej/yw4cPtLS0MGHCBJqbmzGZTOrM0OfPn1sNe7979y4LFiygvLycyZMnk5aWRkhICGFhYZjNZqqqqtDpdBw7doyTJ09y7dq1P/iJ+87f35/S0lLa2tpwcnKiqKhInnDxfzCbzWzatImCggJ8fX3R6/XEx8dbDSG4dOkSBoMBg8HAnTt32LhxI3fu3LFh1OJvI1eY4rfrugr9tkw6dOhQdVRaQEAAxcXFeHl5ERAQgMViYfny5dy+fZsVK1ZQUVFBc3MzN27coLa2lvnz51NWVkZwcDD+/v4oioK3tzfPnj3j3bt3lJaWUllZybZt27q970Axc+ZMEhMTmTZtGjqdDovFwvr1620dlt0oKysjKCiIcePG4ejoyLJly8jNzbU6Jjc3l5SUFDQaDbNmzaK1tXVAVx2E/ZErTPHH/Kg5p2ufm5uburyio6OD9vZ28vPz+eeff/Dy8iIzMxMPDw8ePnxIYmKi1TMSNRoNrq6ubN68mdTUVPUhvwO1KSgjI4OMjAxbh2GXGhoa1O5sAF9f325Xjz0d09DQoI7HE+JXScIUNtU1b3bw4MFotVq0Wi1Hjx4FOhtl3N3defnyJR4eHowePVp9DUB0dDQbNmzAxcWFhIQEm30Ge7NmzRouXLjA6NGjqa6uBuDNmzcsXbqUuro6xo4dy6lTp3B3d7dxpF/1VDX4/h+j3hwjxK+QkqywKY1GYzUuT1EUtZHI3d0di8WCn58fWVlZ6ni+ruYjR0dHdRB812vFz61evVqdB9xl9+7dREZGYjAYiIyMHHCPQ/P19eXly5fq7/X19d3WsfbmGCF+hSRMMaB8n0AHDRqkJsKuJqOmpiaqq6u5du0aGzduZNKkSeprxc/Nnj2bkSNHWm3Lzc1l1apVAKxatUqdwjRQ6PV6DAYDtbW1tLe3k5OTQ3x8vNUx8fHxHD9+HEVR1CVJUo4Vv5OUZMWA930ivHfvHunp6WzZsoWVK1cCXycBib559eqVmly8vb1pbm62cUTWHBwcOHjwINHR0ZjNZtasWUNISIi6znfDhg3ExsaSl5dHUFAQzs7O8oQa8dvJA6SF3ZIk2Xd1dXXExcWp9zBHjBhBa2urut/d3Z2WlhZbhSeErfV4YpGSrLBbkix/H09PT3UJhtFoVBushBBfScIUQhAfH092djYA2dnZLFy40MYRCTHwSElWiH+Z5ORkSkpKMJlMeHp6kpGRwaJFi0hKSuLFixf4+/tz+vTpbo1BQvyL9Fi+koQphBBCWJN7mEIIIURf/WxZiXRVCCGEEMgVphBCCNErkjCFEEKIXpCEKYQQQvSCJEwhhBCiFyRhCiGEEL0gCVMIIYTohf8Ae0grjPe7ZuwAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/markov_prop_7_1.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "α = 0.6\n", + "λ = 0.5\n", + "b = 10\n", + "n = b + 1\n", + "states = np.arange(n)\n", + "I = np.identity(n)\n", + "\n", + "K = np.zeros((n, n))\n", + "K[0, -1] = 1\n", + "for i in range(1, n):\n", + " for j in range(0, i):\n", + " if j == 0:\n", + " K[i, j] = (1 - α)**(i-1)\n", + " else:\n", + " K[i, j] = α * (1 - α)**(i-j-1)\n", + "\n", + "\n", + "def P_t(ψ, t):\n", + " return ψ @ expm(t * λ * (K - I))\n", + "\n", + "def plot_distribution_dynamics(ax, ψ_0, steps=200, step_size=0.1):\n", + " ψ = ψ_0\n", + " t = 0.0\n", + " colors = cm.jet_r(np.linspace(0.0, 1, steps))\n", + "\n", + " for i in range(steps):\n", + " ax.bar(states, ψ, zs=t, zdir='y', \n", + " color=colors[i], alpha=0.8, width=0.4)\n", + " ψ = P_t(ψ, t=step_size)\n", + " t += step_size\n", + "\n", + " ax.set_xlabel('inventory')\n", + " ax.set_ylabel('$t$')\n", + "\n", + "\n", + "ψ_0 = binom.pmf(states, n, 0.25)\n", + "fig = plt.figure(figsize=(8, 6))\n", + "ax = fig.add_subplot(111, projection='3d')\n", + "plot_distribution_dynamics(ax, ψ_0)\n", + "\n", + "from myst_nb import glue\n", + "glue(\"flow_fig\", fig, display=False)\n", + "\n", + "plt.show()" + ] + } + ], + "metadata": { + "jupytext": { + "formats": "ipynb,md:myst", + "text_representation": { + "extension": ".md", + "format_name": "myst", + "format_version": "0.9", + "jupytext_version": "1.5.0" + } + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.7" + }, + "source_map": [ + 13, + 78, + 91, + 563, + 600, + 604, + 617, + 1041 + ] + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file diff --git a/_sources/markov_prop.md b/_sources/markov_prop.md new file mode 100644 index 0000000..e79b3de --- /dev/null +++ b/_sources/markov_prop.md @@ -0,0 +1,1086 @@ +--- +jupytext: + formats: ipynb,md:myst + text_representation: + extension: .md + format_name: myst + format_version: '0.9' + jupytext_version: 1.5.0 +kernelspec: + display_name: Python 3 + language: python + name: python3 +--- + +# The Markov Property + +## Overview + + +A continuous time stochastic process is said to have the Markov property if +that the past and future are independent given the current state. + +(A more formal definition is provided below.) + +As we will see, the Markov property imposes a large amount of structure on +continuous time processes. + +This structure leads to elegant and powerful results on +evolution and dynamics. + +At the same time, the Markov property is general enough to cover many applied +problems, as described in {doc}`the introduction `. + + + +### Setting + +In this lecture, the state space where dynamics +evolve will be a [countable set](https://en.wikipedia.org/wiki/Countable_set), +denoted henceforth by $S$, with typical elements $x, y$. + +(Note that "countable" is understood to include finite.) + +Regarding notation, in what follows, $\sum_{x \in S}$ is abbreviated to +$\sum_x$, the supremum $\sup_{x \in S}$ is abbreviated to $\sup_x$ and so on. + +A **distribution** on $S$ is a function $\phi$ from $S$ to $\RR_+$ with +$\sum_x \phi(x) = 1$. + +Let $\dD$ denote the set of all distributions on $S$. + +To economize on terminology, we define a **matrix** $A$ on $S$ to be a map +from $S \times S$ to $\RR$. + +When $S$ is finite, this reduces to the usual notion of a matrix, and, +whenever you see expressions such as $A(x,y)$ below, you can +mentally identify them with more familiar matrix +notation, such as $A_{ij}$, if you wish. + +The product of two matrices $A$ and $B$ is defined by + +$$ + (A B)(x, y) = \sum_z A(x, z) B(z, y) + \qquad ((x, y) \in S \times S) +$$ (kernprod) + +If $S$ is finite, then this is just ordinary matrix multiplication. + +In statements involving matrix algebra, we *always treat distributions as row +vectors*, so that, for $\phi \in \dD$ and given matrix $A$, + +$$ + (\phi A)(y) = \sum_x \phi(x) A(x, y) +$$ + +We will use the following imports + +```{code-cell} ipython3 +import numpy as np +import scipy as sp +import matplotlib.pyplot as plt + +import quantecon as qe +from numba import njit + +from scipy.linalg import expm +from scipy.stats import binom + +from matplotlib import cm +from mpl_toolkits.mplot3d import Axes3D +``` + +## Markov Processes + +We now introduce the definition of Markov processes, first reviewing the +discrete case and then shifting to continuous time. + + + + +(finstatediscretemc)= +### Discrete Time, Finite State + +The simplest Markov processes are those with a discrete time parameter and finite state space. + +Assume for now that $S$ has $n$ elements and let $P$ be a **Markov matrix**, +which means that $P(x,y) \geq 0$ and $\sum_y P(x,y) = 1$ for all $x$. + +In applications, $P(x, y)$ represents the probability of transitioning from $x$ to +$y$ in one step. + +A **Markov chain** $(X_t)_{t \in \ZZ_+}$ on $S$ with Markov +matrix $P$ is a sequence of random variables satisfying + +$$ + \PP\{X_{t+1} = y \,|\, X_0, X_1, \ldots, X_t \} = P (X_t, y) +$$ (markovpropd) + +with probability one for all $y \in S$ and any $t \in \ZZ_+$. + +In addition to connecting probabilities to the Markov matrix, +{eq}`markovpropd` says that the process depends on its history only through +the current state. + +We [recall that](https://python.quantecon.org/finite_markov.html), if $X_t$ +has distribution $\phi$, then $X_{t+1}$ has distribution $\phi P$. + +Since $\phi$ is understood as a row vector, the meaning is + +$$ + (\phi P)(y) = \sum_x \phi(x) P(x, y) + \qquad (y \in S) +$$ (update_rule) + +(jdfin)= +#### The Joint Distribution + +In general, for given Markov matrix $P$, there can be many Markov chains +$(X_t)$ that satisfy {eq}`markovpropd`. + +This is due to the more general observation that, for a given distribution +$\phi$, we can construct many random variables having distribution $\phi$. + +(The exercises below ask for one example.) + +Hence $P$ is, in a sense, a more primitive object than $(X_t)$. + +There is another way to see the fundamental importance of $P$, which is by +constructing the joint distribution of $(X_t)$ from $P$. + +Let $S^\infty$ represent the space of $S$-valued sequences $(x_0, x_1, x_2, \ldots)$. + +Fix an initial condition $\psi \in \dD$ and a Markov matrix $P$ on $S$. + +The **joint distribution** of a Markov chain $(X_t)$ satisfying +{eq}`markovpropd` and $X_0 \sim \psi$ is the distribution $\mathbf P_\psi$ over +$S^\infty$ such that + +$$ + \PP\{ X_{t_1} = y_1, \ldots, X_{t_k} = y_k \} + = + \mathbf P_\psi\{ (x_t) \in S^\infty \,:\, + x_{t_i} = y_i \text{ for } i = 1, \ldots m\} +$$ (jointdeq) + +for any $m$ positive integers $t_i$ and $m$ elements $y_i$ of the state space $S$. + +(Joint distributions of discrete time processes are uniquely defined by their +values at finite collections of times --- see, for example, Theorem 7.2 of {cite}`walsh2012knowing`.) + +We can construct $\mathbf P_\psi$ by first defining $P_\psi^n$ over +the the finite Cartiesian product $S^{n+1}$ via + +$$ + \mathbf P_\psi^n(x_0, x_1, \ldots, x_n) + = \psi(x_0) + P(x_0, x_1) + \times \cdots \times + P(x_{n-1}, x_n) +$$ (mathjointd) + +For any Markov chain $(X_t)$ satisfying {eq}`markovpropd` and $X_0 \sim \psi$, +the restriction $(X_0, \ldots, X_n)$ has joint distribution $\mathbf +P_\psi^n$. + +This is a solved exercise below. + +The last step is to show that the family $(\mathbf P_\psi^n)$ defined at each +$n \in \NN$ extends uniquely to a distribution $\mathbf P_\psi$ over the +infinite sequences in $S^\infty$. + +That this is true follows from a well known [theorem of Kolmogorov](https://en.wikipedia.org/wiki/Kolmogorov_extension_theorem). + +Hence $P$ defines the joint distribution $\mathbf P_\psi$ when paired with any initial condition $\psi$. + + + +### Extending to Countable State Spaces + +When $S$ is infinite, the same idea carries through. + +Consistent with the finite case, a **Markov matrix** is a map +$P$ from $S \times S$ to $\RR_+$ satisfying + +$$ + \sum_y P(x, y) = 1 + \text{ for all } x \in S +$$ + +The definition of a Markov chain $(X_t)_{t \in \ZZ_+}$ on $S$ with Markov matrix $P$ is exactly as in {eq}`markovpropd`. + +Given Markov matrix $P$ and $\phi \in \dD$, we define $\phi P$ by +{eq}`update_rule`. + +Then, as before, $\phi P$ can be understood as the distribution of +$X_{t+1}$ when $X_t$ has distribution $\phi$. + +The function $\phi P$ is in $\dD$, since, by {eq}`update_rule`, it is +nonnegative and + +$$ + \sum_y (\phi P)(y) + = \sum_y \sum_x P(x, y) \phi(x) + = \sum_x \sum_y P(x, y) \phi(x) + = \sum_x \phi(x) + = 1 +$$ + +(Swapping the order of infinite sums is justified here by the fact that all +elements are nonnegative --- a version of Tonelli's theorem). + +If $P$ and $Q$ are Markov matrices on $S$, then, using the definition in +{eq}`kernprod`, + +$$ + (P Q)(x, y) := \sum_z P(x, z) Q(z, y) +$$ + +It is not difficult to check that $P Q$ is again a Markov matrix on $S$. + +The elements of $P^k$, the $k$-th product of $P$ with itself, give $k$ step transition probabilities. + +For example, we have + +$$ + P^k(x, y) + = (P^{k-j} P^j)(x, y) = \sum_z P^{k-j}(x, z) P^j(z, y) +$$ (kernprodk) + +which is a version of the (discrete time) Chapman-Kolmogorov equation. + +Equation {eq}`kernprodk` can be obtained from the law of total probability: if +$(X_t)$ is a Markov chain with Markov matrix $P$ and initial condition $X_0 = +x$, then + +$$ + \PP\{X_k = y\} + = \sum_z \PP\{X_k = y \,|\, X_j=z\} \PP\{X_j=z\} +$$ + + +All of the {ref}`preceding discussion ` on the connection between $P$ +and the joint distribution of $(X_t)$ when $S$ is finite carries over +to the current setting. + + + + +### The Continuous Time Case + +A **continuous time stochastic process** on $S$ is a collection $(X_t)$ of $S$-valued +random variables $X_t$ defined on a common probability space and indexed by $t +\in \RR_+$. + +Let $I$ be the Markov matrix on $S$ defined by $I(x,y) = \mathbb 1\{x = y\}$. + +A **Markov semigroup** is a family $(P_t)$ of Markov matrices +on $S$ satisfying + +1. $P_0 = I$, +2. $\lim_{t \to 0} P_t(x, y) = I(x,y)$ for all $x,y$ in $S$, and +3. the semigroup property $P_{s + t} = P_s P_t$ for all $s, t \geq 0$. + +The interpretation of $P_t(x, y)$ is the probability of moving from state $x$ +to state $y$ in $t$ units of time. + +As such it is natural that $P_0(x,y) = 1$ if $x=y$ and zero otherwise, which +is condition 1. + +Condition 2 is continuity with respect to $t$, which might seem restrictive +but it is in fact very mild. + +For all practical applications, probabilities do not jump --- although the +chain $(X_t)$ itself can of course jump from state to state as time +goes by.[^footnote1] + +[^footnote1]: On a technical level, right continuity of paths for $(X_t)$ implies condition 2, as proved in Theorem 2.12 of {cite}`liggett2010continuous`. Right continuity of paths allows for jumps, but insists on only finitely many jumps in any bounded interval. + + +The semigroup property in condition 3 is nothing more than a continuous +time version of the Chapman-Kolmogorov equation. + +This becomes clearer if we write it more explicitly as + +$$ + P_{s+t}(x, y) + = \sum_z P_s(x, z) P_t(z, y) +$$ (chapkol_ct2) + +A stochastic process $(X_t)$ is called a (time homogeneous) **continuous time +Markov chain** on $S$ with Markov semigroup $(P_t)$ if + +$$ + \PP\{X_{s + t} = y \,|\, \fF_s \} + = P_t (X_s, y) +$$ (markovprop) + +with probability one for all $y \in S$ and $s, t \geq 0$. + +Here $\fF_s$ is the history $(X_r)_{r \leq s}$ of the process up until +time $s$. + +If you are an economist you might call $\fF_s$ the "information set" at time +$s$. + +If you are familiar with measure theory, you can understand $\fF_s$ as +the $\sigma$-algebra generated by $(X_r)_{r \leq s}$. + +Analogous to the discrete time case, the joint +distribution of $(X_t)$ is determined by its Markov semigroup plus an +initial condition. + +This distribution is defined over the set of all right continuous functions +$\RR_+ \ni t \mapsto x_t \in S$, which we call $rcS$. + +Next one builds finite dimensional distributions over $rcS$ using +expressions similar to {eq}`mathjointd`. + +Finally, the Kolmogorov extension theorem is applied, similar to the discrete +time case. + +Corollary 6.4 of {cite}`le2016brownian` provides full details. + + +### The Canonical Chain + +Given a Markov semigroup $(P_t)$ on $S$, does there always exist a continuous +time Markov chain $(X_t)$ such that {eq}`markovprop` holds? + +The answer is affirmative. + +To illustrate, pick any Markov semigroup $(P_t)$ on $S$ and fix initial +condition $\psi$. + +Next, create the corresponding joint distribution $\mathbf P_\psi$ over +$rcS$, as described above. + +Now, for each $t \geq 0$, let $\pi_t$ be the point evaluation functional on +$rcS$, that maps right continuous function $(x_\tau)$ into its time $t$ value +$x_t$. + +Finally, let $X_t$ be an $S$-valued function on $rcS$ defined at $(x_\tau) \in rcS$ by $\pi_t ( (x_\tau))$. + +In other words, after $\mathbf P_\psi$ picks out some time path $(x_\tau) \in +rcS$, the Markov chain $(X_t)$ simply reports this time path. + +Hence $(X_t)$ automatically has the correct distribution. + +The chain $(X_t)$ constructed in this way is called the **canonical chain** +for the semigroup $(P_t)$ and initial condition $\psi$. + + +### Simulation and Probabilistic Constructions + +While we have answered the existence question in the affirmative, +the canonical construction is quite abstract. + +Moreover, there is little information about how we might simulate such a chain. + +Fortunately, it turns out that there are more concrete ways to build +continuous time Markov chains from the objects that describe their +distributions. + +We will learn about these in a {doc}`later lecture `. + + + +## Implications of the Markov Property + +The Markov property carries some strong implications that are not immediately +obvious. + +Let's take some time to explore them. + +### Example: Failure of the Markov Property + +Let's look at how the Markov property can fail, via an intuitive rather than +formal discussion. + +Let $(X_t)$ be a continuous time stochastic process with state space $S = \{0, 1\}$. + +The process starts at $0$ and updates at follows: + +1. Draw $W$ independently from a fixed Pareto distribution. +1. Hold $(X_t)$ in its current state for $W$ units of time and then switch + to the other state. +1. Go to step 1. + +What is the probability that $X_{s+h} = i$ given both the history $(X_r)_{r \leq s}$ and current information $X_s = i$? + +If $h$ is small, then this is close to the +probability that there are zero switches over the time interval $(s, s+h]$. + +To calculate this probability, it would be helpful to know how long the +state has been at current state $i$. + +This is because the Pareto distribution {ref}`is not memoryless `. + +(With a Pareto distribution, if we know that $X_t$ has been at $i$ for a long +time, then a switch in the near future becomes more likely.) + +As a result, the history prior to $X_s$ is useful for predicting $X_{s+h}$, +even when we know $X_s$. + +Thus, the Markov property fails. + + + +### Restrictions Imposed by the Markov Property + +From the discussion above, we see that, for continuous time Markov chains, +the length of time between jumps must be memoryless. + +Recall that, by {proof:ref}`exp_unique`, the only memoryless +distribution supported on $\RR_+$ is the exponential distribution. + +Hence, a continuous time Markov chain waits at states for an +exponential amount of time and then jumps. + +The way that the new state is chosen must also satisfy the Markov property, +which adds another restriction. + +In summary, we already understand the following about continuous time Markov chains: + +1. Holding times are independent exponential draws. +1. New states are chosen in a ``Markovian'' way, independent of the past given the current state. + +We just need to clarify the details in these steps to have a complete description. + + + + +## Examples of Markov Processes + +Let's look at some examples of processes that possess the Markov property. + +### Example: Poisson Processes + +The Poisson process discussed in our {doc}`previous lecture ` is a +Markov process on state space $\ZZ_+$. + +To obtain the Markov semigroup, we observe that, for $k \geq j$, + +$$ + \PP\{N_{s + t} = k \,|\, N_s = j\} + = \PP\{N_{s + t} - N_s = k - j \,|\, N_s = j\} + = \PP\{N_{s + t} - N_s = k - j\} +$$ + +where the last step is due to independence of increments. + +From stationarity of increments we have + +$$ + \PP\{N_{s + t} - N_s = k - j\} + = \PP\{N_t = k - j\} + = e^{-\lambda t} \frac{ (\lambda t)^{k-j} }{(k-j)!} +$$ + +In summary, the Markov semigroup is + +$$ + P_t(j, k) + = e^{-\lambda t} \frac{ (\lambda t)^{k-j} }{(k-j)!} +$$ (poissemi) + +whenever $j \leq k$ and $P_t(j, k) = 0$ otherwise. + +This chain of equalities was obtained with $N_s = j$ for arbitrary $j$, so we +can replace $j$ with $N_s$ in {eq}`poissemi` to verify the Markov property {eq}`markovprop` for the Poisson process. + +Under {eq}`poissemi`, each $P_t$ is a Markov matrix and $(P_t)$ is a +Markov semigroup. + +The proof of the semigroup property is a solved exercise below.[^footnote2] + +[^footnote2]: In the definition of $P_t$ in {eq}`poissemi`, we use the convention that $0^0 = 1$, which leads to $P_0 = I$ and $\lim_{t \to 0} P_t(j, k) = I(j,k)$ for all $j,k$. These facts, along with the semigroup property, imply that $(P_t)$ is a valid Markov semigroup. + + + + + +(inventory_dynam)= +## A Model of Inventory Dynamics + + +Let $X_t$ be the inventory of a firm at time $t$, taking values in the +integers $0, 1, \ldots, b$. + +If $X_t > 0$, then a customer arrives after $W$ +units of time, where $W \sim E(\lambda)$ for some fixed $\lambda > 0$. + +Upon arrival, each customer purchases $\min\{U, X_t\}$ units, where $U$ is an +IID draw from the geometric distribution started at 1 rather than 0: + +$$ + \PP\{U = k\} = (1-\alpha)^{k-1} \alpha + \qquad (k = 1, 2, \ldots, \; \alpha \in (0, 1)) +$$ + +If $X_t = 0$, then no customers arrive and the firm places an order for $b$ units. + +The order arrives after a delay of $D$ units of time, where $D \sim E(\lambda)$. + +(We use the same $\lambda$ here just for convenience, to simplify the exposition.) + +### Representation + +The inventory process jumps to a new value either when a new customer arrives +or when new stock arrives. + +Between these arrival times it is constant. + +Hence, to track $X_t$, it is enough to track the jump times and the new values +taken at the jumps. + +In what follows, we denote the jump times by $\{J_k\}$ and the values at jumps +by $\{Y_k\}$. + +Then we construct the state process via + +$$ + X_t = \sum_{k \geq 0} Y_k \mathbb 1\{J_k \leq t < J_{k+1}\} + \qquad (t \geq 0) +$$ (xfromy) + + + +### Simulation + +Let's simulate this process, starting at $X_0 = 0$. + +As above, + +* $J_k$ is the time of the $k$-th jump (up or down) in inventory. +* $Y_k$ is the size of the inventory after the $k$-th jump. +* $(X_t)$ is defined from these objects via {eq}`xfromy`. + +Here's a function that generates and returns one path $t \mapsto X_t$. + +(We are not aiming for computational efficiency at this stage.) + +```{code-cell} ipython3 +def sim_path(T=10, seed=123, λ=0.5, α=0.7, b=10): + """ + Generate a path for inventory starting at b, up to time T. + + Return the path as a function X(t) constructed from (J_k) and (Y_k). + """ + + J, Y = 0, b + J_vals, Y_vals = [J], [Y] + np.random.seed(seed) + + while True: + W = np.random.exponential(scale=1/λ) # W ~ E(λ) + J += W + J_vals.append(J) + if J >= T: + break + # Update Y + if Y == 0: + Y = b + else: + U = np.random.geometric(α) + Y = Y - min(Y, U) + Y_vals.append(Y) + + Y_vals = np.array(Y_vals) + J_vals = np.array(J_vals) + + def X(t): + if t == 0.0: + return Y_vals[0] + else: + k = np.searchsorted(J_vals, t) + return Y_vals[k-1] + + return X +``` + +Let's plot the process $(X_t)$. + +```{code-cell} ipython3 +T = 20 +X = sim_path(T=T) + +grid = np.linspace(0, T, 100) + +fig, ax = plt.subplots() +ax.step(grid, [X(t) for t in grid], label="$X_t$") + +ax.set(xlabel="time", ylabel="inventory") + +ax.legend() +plt.show() +``` + +As expected, inventory falls and then jumps back up to $b$. + + + +### The Embedded Jump Chain + +In models such as the one described above, the embedded discrete time +process $(Y_k)$ is called the "embedded jump chain". + +It is easy to see that $(Y_k)$ is discrete time finite state Markov chain. + +Its Markov matrix $K$ is +given by $K(x, y) = \mathbb 1\{y=b\}$ when $x=0$ and, when $0 < x \leq b$, + +$$ + K(x, y) + = + \begin{cases} + \mathbb 0 & \text{ if } y \geq x + \\ + \PP\{x - U = y\} = (1-\alpha)^{x-y-1} \alpha + & \text{ if } 0 < y < x + \\ + \PP\{U \geq x\} = (1-\alpha)^{x-1} + & \text{ if } y = 0 + \end{cases} +$$ (ijumpkern) + + + + +### Markov Property + +The inventory model just described has the Markov property precisely because + +1. the jump chain $(Y_k)$ is Markov in discrete time and +1. the holding times are independent exponential draws. + +Rather than providing more details on these points here, let us first describe +a more general setting where the arguments will be clearer and more useful. + + + +## Jump Processes with Constant Rates + +The examples we have focused on so far are special cases of Markov processes +with constant jump intensities. + +This processes turn out to be very representative (although the constant jump intensity will later be relaxed). + +Let's now summarize the model and its properties. + + +### Construction + +The data for a Markov process on $S$ with constant jump rates are + +* a parameter $\lambda > 0$ called the **jump rate**, which governs the jump + intensities and +* a Markov matrix $K$ on $S$, called the **jump matrix**. + +To run the process we also need an initial condition $\psi \in \dD$. + +The process $(X_t)$ is constructed by holding at each state for an +exponential amount of time, with rate $\lambda$, and then updating to a +new state via $K$. + +In more detail, the construction is + +```{proof:algorithm} Constant Rate Jump Chain + +**Inputs** $\psi \in \dD$, positive constant $\lambda$, Markov matrix $K$ + +**Outputs** Markov chain $(X_t)$ + +1. draw $Y_0$ from $\psi$ +1. set $k = 1$ and $J_0 = 0$ +1. draw $W_k$ from Exp$(\lambda)$ and set $J_k = J_{k-1} + W_k$ +1. set $X_t = Y_{k-1}$ for all $t$ such that $J_{k-1} \leq t < J_k$. +1. draw $Y_k$ from $K(Y_{k-1}, \cdot)$ +1. set $k = k+1$ and go to step 3. + +``` + +An alternative, more parsimonious way to express the same process is to take + +* $(N_t)$ to be a Poisson process with rate $\lambda$ and +* $(Y_k)$ to be a discrete time Markov chain with Markov matrix $K$ + +and then set + +$$ + X_t := Y_{N_t} \text{ for all } t \geq 0 +$$ + +As before, the discrete time process $(Y_k)$ is called the **embedded jump chain**. + +(Not to be confused with $(X_t)$, which is often called a "jump process" or +"jump chain" due to the fact that it changes states with jumps.) + +The draws $(W_k)$ are called the **wait times** or **holding times**. + + +### Examples + +The Poisson process with rate $\lambda$ is a jump process on $S = \ZZ_+$. + +The holding times are obviously exponential with constant rate $\lambda$. + +The jump matrix is just $K(i, j) = \mathbb 1\{j = i+1\}$, so that the state +jumps up by one at every $J_k$. + +The inventory model is also a jump process with constant rate $\lambda$, this +time on $S = \{0, 1, \ldots, b\}$. + +The jump matrix was given in {eq}`ijumpkern`. + + + + + +### Markov Property + +Let's show that the jump process $(X_t)$ constructed above satisfies the +Markov property, and obtain the Markov semigroup at the same time. + +We will use two facts: + +* the jump chain $(Y_k)$ has the Markov property in discrete + time and +* the Poisson process has stationary independent increments. + +From these facts it is intuitive that the distribution of $X_{t+s}$ given +the whole history $\fF_s = \{ (N_r)_{r \leq s}, (Y_k)_{k \leq N_s} \}$ +depends only on $X_s$. + +Indeed, if we know $X_s$, then we can simply + +* {ref}`restart ` the Poisson process from $N_s$ and then +* starting from $X_s = Y_{N_s}$, update the embedded jump chain $(Y_k)$ using $K$ each time a new jump occurs. + +Let's write this more mathematically. + +Fixing $y \in S$ and $s, t \geq 0$, we have + + +$$ + \PP\{X_{s + t} = y \,|\, \fF_s \} + = \PP\{Y_{N_{s + t}} = y \,|\, \fF_s \} + = \PP\{Y_{N_s + N_{s + t} - N_s} = y \,|\, \fF_s \} +$$ + +{ref}`Recalling ` that $N_{s + t} - N_s$ is Poisson distributed with rate $t \lambda$, independent of the history $\fF_s$, we can write the display above as + +$$ + \PP\{X_{s + t} = y \,|\, \fF_s \} + = + \sum_{k \geq 0} + \PP\{Y_{N_s + k} = y \,|\, \fF_s \} + \frac{(t \lambda )^k}{k!} e^{-t \lambda} +$$ + +Because the embedded jump chain is Markov with Markov matrix $K$, we can simplify further to + +$$ + \PP\{X_{s + t} = y \,|\, \fF_s \} + = \sum_{k \geq 0} + K^k(Y_{N_s}, y) \frac{(t \lambda )^k}{k!} e^{-t \lambda} + = K^k(X_s, y) \frac{(t \lambda )^k}{k!} e^{-t \lambda} +$$ + +Since the expression above depends only on $X_s$, +we have proved that $(X_t)$ has the Markov property. + + +(consjumptransemi)= +### Transition Semigroup + +The Markov semigroup can be obtained from our final result, conditioning +on $X_s = x$ to get + +$$ + P^t(x, y) = \PP\{X_{s + t} = y \,|\, X_s = x \} + = e^{-t \lambda} \sum_{k \geq 0} + K^k(x, y) \frac{(t \lambda )^k}{k!} +$$ + +If $S$ is finite, we can write this in matrix form and use the definition of +the [matrix exponential](https://en.wikipedia.org/wiki/Matrix_exponential) to +get + +$$ + P^t + = e^{-t \lambda} + \sum_{k \geq 0} + \frac{(t \lambda K)^k}{k!} + = e^{-t \lambda} e^{t \lambda K} + = e^{t \lambda (K - I)} +$$ + +This is a simple and elegant representation of the Markov semigroup that +makes it easy to understand and analyze distribution dynamics. + +For example, if $X_0$ has distribution $\psi$, then $X_t$ has distribution + +$$ + \psi P_t = \psi e^{t \lambda (K - I)} +$$ (distflowconst) + +We just need to plug in $\lambda$ and $K$ to obtain the entire flow $t \mapsto \psi P_t$. + +We will soon extend this representation to the case where $S$ is infinite. + + +(invdistflows)= +## Distribution Flows for the Inventory Model + +Let's apply these ideas to the inventory model described above. + +We fix + +* the parameters $\alpha$, $b$ and $\lambda$ in the inventory model and +* an initial condition $X_0 \sim \psi_0$, where $\psi_0$ is an arbitrary +distribution on $S$. + +The state $S$ is set to $\{0, \ldots, b\}$ and the matrix $K$ is defined by +{eq}`ijumpkern`. + +Now we run time forward. + +We are interesting in computing the flow of distributions $t \mapsto \psi_t$, +where $\psi_t$ is the distribution of $X_t$. + +According to the theory developed above, we have two options: + +Option 1 is to use simulation. + +The first step is to simulate many independent observations the process $(X_t^m)_{m=1}^M$. + +(Here $m$ indicates simulation number $m$, which you might think of as the outcome +for firm $m$.) + +Next, for any given $t$, we define $\hat \psi_t \in \dD$ as the +histogram of observations at time $t$, or, equivalently the cross-sectional +distribution at $t$: + +$$ + \hat \psi_t(x) := \frac{1}{M} \sum_{m=1}^M \mathbb 1\{X_t = x\} + \qquad (x \in S) +$$ + +Then $\hat \psi_t(x)$ will be close to $\PP\{X_t = x\}$ by the law of +large numbers. + +In other words, in the limit we recover $\psi_t$. + +Option 2 is to insert the parameters into the right hand side of {eq}`distflowconst` +and compute $\psi_t$ as $\psi_0 P_t$. + +The figure below is created using option 2, with $\alpha = 0.6$, $\lambda = 0.5$ and $b=10$. + +For the initial distribution we pick a binomial distribution. + + +Since we cannot compute the entire uncountable flow $t \mapsto \psi_t$, we +iterate forward 200 steps at time increments $h=0.1$. + +In the figure, hot colors indicate initial conditions and early dates (so that the +distribution "cools" over time) + +```{glue:figure} flow_fig +:name: "flow_fig" + +Probability flows for the inventory model. +``` + +In the (solved) exercises you will be asked to try to reproduce this figure. + + +## Exercises + +### Exercise 1 + +Consider the binary (Bernoulli) distribution where outcomes $0$ and $1$ each have +probability $0.5$. + +Construct two different random variables with this distribution. + + + +### Exercise 2 + +Show by direct calculation that the Poisson matrices $(P_t)$ defined in +{eq}`poissemi` satisfy the semigroup property {eq}`chapkol_ct2`. + +Hints + +* Recall that $P_t(j, k) = 0$ whenever $j > k$. +* Consider using the [binomial formula](https://en.wikipedia.org/wiki/Binomial_theorem). + + +### Exercise 3 + +Consider the distribution over $S^{n+1}$ previously shown in {eq}`mathjointd`, which is + +$$ + \mathbf P_\psi^n(x_0, x_1, \ldots, x_n) + = \psi(x_0) + P(x_0, x_1) + \times \cdots \times + P(x_{n-1}, x_n) +$$ + +Show that, for any Markov chain $(X_t)$ satisfying {eq}`markovpropd` +and $X_0 \sim \psi$, the restriction $(X_0, \ldots, X_n)$ has joint +distribution $\mathbf P_\psi^n$. + +### Exercise 4 + +Try to produce your own version of the figure {ref}`flow_fig`. + +The initial condition is ``ψ_0 = binom.pmf(states, n, 0.25)`` where ``n = b + 1``. + + +## Solutions + +### Solution to Exercise 1 + +This is easy. + +One example is to take $U$ to be uniform on $(0, 1)$ and set $X=0$ if $U < +0.5$ and $1$ otherwise. + +Then $X$ has the desired distribution. + +Alternatively, we could take $Z$ to be standard normal and set $X=0$ if $Z < +0$ and $1$ otherwise. + + +### Solution to Exercise 2 + +Fixing $s, t \in \RR_+$ and $j \leq k$, we have + +$$ +\begin{aligned} + \sum_{i \geq 0} P_s(j, i) P_t(i, k) + & = + e^{-\lambda (s+t)} + \sum_{j \leq i \leq k} + \frac{ (\lambda s)^{i-j} }{(i-j)!} + \frac{ (\lambda t)^{k-i} }{(k-i)!} + \\ + & = + e^{-\lambda (s+t)} \lambda^{k-j} + \sum_{0 \leq \ell \leq k-j} + \frac{ s^\ell }{\ell!} + \frac{ t^{k-j - \ell} }{(k-j - \ell)!} + \\ + & = + e^{-\lambda (s+t)} \lambda^{k-j} + \sum_{0 \leq \ell \leq k-j} + \binom{k-j}{\ell} + \frac{s^\ell t^{k-j - \ell}}{(k-j)!} +\end{aligned} +$$ + +Applying the binomial formula, we can write this as + +$$ + \sum_{i \geq 0} P_s(j, i) P_t(i, k) + = + e^{-\lambda (s+t)} + \frac{(\lambda (s + t))^{k-j}}{(k-j)!} + = P_{s+t}(j, k) +$$ + +Hence {eq}`chapkol_ct2` holds, and the semigroup property is satisfied. + + +### Solution to Exercise 3 + +Let $(X_t)$ be a Markov chain satisfying {eq}`markovpropd` and $X_0 \sim \psi$. + +When $n=0$, we have $\mathbf P_\psi^n = \mathbf P_\psi^0 = \psi$, and this +agrees with the distribution of the restriction $(X_0, \ldots, X_n) = (X_0)$. + +Now suppose the same is true at arbitrary $n-1$, in the sense that +the distribution of $(X_0, \ldots, X_{n-1})$ is equal to $\mathbf P_\psi^{n-1}$ as +defined above. + +Then + +$$ + \PP \{X_0 = x_0, \ldots, X_n = x_n\} + = \PP \{X_n = x_n \,|\, X_0 = x_0, \ldots, X_{n-1} = x_{n-1} \} + \\ + \times \PP \{X_0 = x_0, \ldots, X_{n-1} = x_{n-1}\} +$$ + +From the Markov property and the induction hypothesis, the right hand side is + +$$ + P (x_{n-1}, x_n ) + \mathbf P_\psi^n(x_0, x_1, \ldots, x_{n-1}) + = + P (x_n, x_{n+1} ) + \psi(x_0) + P(x_0, x_1) + \times \cdots \times + P(x_{n-1}, x_{n-1}) +$$ + +The last expression equals $\mathbf P_\psi^n$, which concludes the proof. + + +### Solution to Exercise 4 + +Here is one approach. + +(The statements involving ``glue`` are specific to this book and can be deleted +by most readers. They store the output so it can be displayed elsewhere.) + +```{code-cell} ipython3 +α = 0.6 +λ = 0.5 +b = 10 +n = b + 1 +states = np.arange(n) +I = np.identity(n) + +K = np.zeros((n, n)) +K[0, -1] = 1 +for i in range(1, n): + for j in range(0, i): + if j == 0: + K[i, j] = (1 - α)**(i-1) + else: + K[i, j] = α * (1 - α)**(i-j-1) + + +def P_t(ψ, t): + return ψ @ expm(t * λ * (K - I)) + +def plot_distribution_dynamics(ax, ψ_0, steps=200, step_size=0.1): + ψ = ψ_0 + t = 0.0 + colors = cm.jet_r(np.linspace(0.0, 1, steps)) + + for i in range(steps): + ax.bar(states, ψ, zs=t, zdir='y', + color=colors[i], alpha=0.8, width=0.4) + ψ = P_t(ψ, t=step_size) + t += step_size + + ax.set_xlabel('inventory') + ax.set_ylabel('$t$') + + +ψ_0 = binom.pmf(states, n, 0.25) +fig = plt.figure(figsize=(8, 6)) +ax = fig.add_subplot(111, projection='3d') +plot_distribution_dynamics(ax, ψ_0) + +from myst_nb import glue +glue("flow_fig", fig, display=False) + +plt.show() +``` diff --git a/_sources/memoryless.ipynb b/_sources/memoryless.ipynb new file mode 100644 index 0000000..2c917da --- /dev/null +++ b/_sources/memoryless.ipynb @@ -0,0 +1,613 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Memoryless Distributions\n", + "\n", + "\n", + "## Overview\n", + "\n", + "\n", + "Markov processes are, by definition, forgetful.\n", + "\n", + "In particular, for any Markov processes, the distribution over future outcomes\n", + "depends only on the current state, rather than the entire history.\n", + "\n", + "In the case of continuous time Markov chains, which jump between discrete\n", + "states, this requires that the amount of elapsed time since the last jump is\n", + "not helpful in predicting the timing of the next jump.\n", + "\n", + "In other words, the jump times are \"memoryless\".\n", + "\n", + "It is remarkable that the only distribution on $\\RR_+$ with this\n", + "property is the exponential distribution.\n", + "\n", + "Similarly, the only memoryless distribution on $\\ZZ_+$ is the geometric\n", + "distribution.\n", + "\n", + "This lecture tries to clarify these ideas.\n", + "\n", + "We will use the following imports:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "import quantecon as qe\n", + "from numba import njit\n", + "from scipy.special import factorial, binom" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The Geometric Distribution\n", + "\n", + "Consider betting on a roulette wheel and suppose that red has come up four times in a row.\n", + "\n", + "Since five reds in a row is an unlikely event, many people instinctively feel\n", + "that black is more likely on the fifth spin --- \"Surely black will come up this time!\"\n", + "\n", + "But rational thought tells us such instincts are wrong: the four previous reds make no\n", + "difference to the outcome of the next spin.\n", + "\n", + "(Many casinos offer an unlimited supply of free alcoholic beverages in order to discourage this kind of rational analysis.)\n", + "\n", + "A mathematical restatement of this phenomenon is: the geometric distribution is memoryless.\n", + "\n", + "\n", + "### Memorylessness\n", + "\n", + "Let $X$ be a random variable supported on the nonnegative integers $\\ZZ_+$.\n", + "\n", + "We say that $X$ is [geometrically distributed](https://en.wikipedia.org/wiki/Geometric_distribution) if, for some $\\theta$ satisfying $0 \\leq \\theta \\leq 1$, \n", + "\n", + "$$ \n", + " \\PP\\{X = k\\} = (1-\\theta)^k \\theta \n", + " \\qquad (k = 0, 1, \\ldots)\n", + "$$ (geodist)\n", + "\n", + "An example can be constructed from the discussion of the roulette wheel above.\n", + "\n", + "Suppose that, \n", + "\n", + "* the outcome of each spin is either red or black, \n", + "* spins are labeled by $0, 1, 2, \\ldots$,\n", + "* on each spin, black occurs with probability $\\theta$ and\n", + "* outcomes across spins are independent.\n", + "\n", + "Then {eq}`geodist` is the probability that the first occurrence of black is at spin $k$.\n", + "\n", + "(The outcome \"black\" fails $k$ times and then succeeds.)\n", + "\n", + "Consistent with our discussion in the introduction, the geometric distribution\n", + "is memoryless.\n", + "\n", + "For example, given any nonnegative integer $m$, we have\n", + "\n", + "$$\n", + " \\PP \\{X = m + 1 \\,|\\, X > m \\} = \\theta\n", + "$$ (memgeo)\n", + "\n", + "In other words, regardless of how long we have seen only red outcomes, the\n", + "probability of black on the next spin is the same as the unconditional\n", + "probability of getting black on the very first spin.\n", + "\n", + "To show this, we note that the left hand side is\n", + "\n", + "$$\n", + " \\frac{ \\PP \\{X = m + 1 \\text{ and } X > m \\} }\n", + " {\\PP \\{X \\geq m\\}}\n", + " =\n", + " \\frac{ \\PP \\{X = m + 1 \\} }\n", + " {\\PP \\{X > m\\}}\n", + " = \\frac{ (1-\\theta)^{m+1} \\theta }\n", + " {\\sum_{k > m} (1-\\theta)^k \\theta }\n", + "$$\n", + "\n", + "The right hand side simplifies to $\\theta$, completing the proof of {eq}`memgeo`.\n", + "\n", + "\n", + "\n", + "## The Exponential Distribution\n", + "\n", + "\n", + "Later, when we construct continuous time Markov chains, we will need to\n", + "specify the distribution of the holding times, which are the time intervals\n", + "between jumps.\n", + "\n", + "As discussed above (and again below), the holding time distribution must be\n", + "memoryless, so that the chain satisfies the Markov property.\n", + "\n", + "While the geometric distribution is memoryless, its discrete support makes it\n", + "a poor fit for the continuous time case.\n", + "\n", + "Hence we turn to the [exponential distribution](https://en.wikipedia.org/wiki/Exponential_distribution), which is supported on $\\RR_+$.\n", + "\n", + "A random variable $Y$ on $\\RR_+$ is called **exponential with rate $\\lambda$**, denoted by $Y \\sim \\Exp(\\lambda)$, if\n", + "\n", + "$$\n", + " \\PP\\{Y > y\\} = e^{-\\lambda y}\n", + " \\qquad (y \\geq 0)\n", + "$$\n", + "\n", + "\n", + "\n", + "(geomtoexp)=\n", + "### From Geometric to Exponential\n", + "\n", + "The exponential distribution can be regarded as the \"limit\" of the geometric\n", + "distribution.\n", + "\n", + "To illustrate, let us suppose that \n", + "\n", + "* customers enter a shop at discrete times $t_0, t_1, \\ldots$\n", + "* these times are evenly spaced, so that $h = t_{i+1} - t_i$ for some $h > 0$ and all $i \\in \\ZZ_+$\n", + "* at each $t_i$, either zero or one customers enter (no more because $h$ is small) \n", + "* entry at each $t_i$ occurs with probability $\\lambda h$ and is independent over $i$.\n", + "\n", + "The fact that the entry probability is proportional to $h$ is important in\n", + "what follows.\n", + "\n", + "You can imagine many customers passing by the shop, each entering\n", + "independently.\n", + "\n", + "If we halve the time interval, then we also halve the probability that a\n", + "customer enters.\n", + "\n", + "Let \n", + "\n", + "* $Y$ be the time of the first arrival at the shop, \n", + "* $t$ be a given positive number and\n", + "* $i(h)$ be the largest integer such that $t_{i(h)} \\leq t$.\n", + "\n", + "Note that, as $h \\to 0$, the grid becomes finer and $t_{i(h)} = i(h) h \\to t$.\n", + "\n", + "Writing $i(h)$ as $i$ and using the geometric distribution, the probability that \n", + "the first arrival occurs after $t_{i}$ is $(1-\\lambda h)^{i}$.\n", + "\n", + "Hence \n", + "\n", + "$$\n", + " \\PP\\{Y > t_{i} \\}\n", + " = (1-\\lambda h)^i\n", + " = \\left( 1- \\frac{\\lambda i h}{i} \\right)^i\n", + "$$\n", + "\n", + "Using the fact that $e^x = \\lim_{i \\to \\infty}(1 + x/i)^i$ for all $x$ and $i\n", + "h = t_i \\to t$, we obtain, for large $i$,\n", + "\n", + "$$\n", + " \\PP\\{Y > t\\}\n", + " \\approx\n", + " e^{- \\lambda t}\n", + "$$\n", + "\n", + "In this sense, the exponential is the limit of the geometric distribution.\n", + "\n", + "\n", + "### Memoryless Property of the Exponential Distribution\n", + "\n", + "The exponential distribution is the only memoryless distribution supported on $\\RR_+$, as the next theorem attests.\n", + "\n", + "```{proof:theorem} Characterization of the Exponential Distribution\n", + ":label: exp_unique\n", + "\n", + "If $X$ is a random variable supported on $\\RR_+$, then there exists a\n", + "$\\lambda > 0$ such that $X \\sim \\Exp(\\lambda)$ if and only if, for all\n", + "positive $s, t$,\n", + "\n", + "$$\n", + " \\PP \\{X > s + t \\,|\\, X > s \\} = \\PP \\{X > t\\}\n", + "$$ (memexpo)\n", + "\n", + "```\n", + "\n", + "```{proof:proof}\n", + "\n", + "To see that {eq}`memexpo` holds when $X$ is exponential with rate $\\lambda$,\n", + "fix $s, t > 0$ and observe that\n", + "\n", + "$$\n", + " \\frac{ \\PP \\{X > s + t \\text{ and } X > s \\} }\n", + " {\\PP \\{X > s\\}}\n", + " =\n", + " \\frac{ \\PP \\{X > s + t \\} }\n", + " {\\PP \\{X > s\\}}\n", + " = \\frac{e^{-\\lambda s - \\lambda t}}{e^{-\\lambda s}}\n", + " = e^{-\\lambda t}\n", + "$$\n", + "\n", + "To see that the converse holds, let $X$ be a random variable supported on $\\RR_+$\n", + "such that {eq}`memexpo` holds.\n", + "\n", + "The \"exceedance\" function $f(s) := \\PP\\{X > s\\}$ then has three properties:\n", + "\n", + "1. $f$ is decreasing on $\\RR_+$,\n", + "1. $0 < f(t) < 1$ for all $t > 0$,\n", + "1. $f(s + t) = f(s) f(t)$ for all $s, t > 0$.\n", + "\n", + "The first property is common to all exceedance functions, the second is due to\n", + "the fact that $X$ is supported on all of $\\RR_+$, and the\n", + "third is {eq}`memexpo`.\n", + "\n", + "From these three properties we will show that\n", + "\n", + "$$\n", + " f(t) = f(1)^t \\;\\; \\forall \\, t \\geq 0\n", + "$$ (implex)\n", + "\n", + "This is sufficient to prove the claim because then $\\lambda := - \\ln f(1)$ is a positive real number (by property 2) and, moreover,\n", + "\n", + "$$ \n", + " f(t) \n", + " = \\exp( \\ln ( f(1) ) t) \n", + " = \\exp( - \\lambda t) \n", + "$$\n", + "\n", + "To see that {eq}`implex` holds, fix positive integers $m,n$.\n", + "\n", + "We can use property 3 to obtain both\n", + "\n", + "$$\n", + " f(m/n) = f(1/n)^m\n", + " \\quad \\text{and} \\quad\n", + " f(1) = f(1/n)^n\n", + "$$\n", + "\n", + "It follows that $f(m/n)^n = f(1/n)^{m n} = f(1)^m$ and, raising to the power\n", + "of $1/n$, we get {eq}`implex` when $t=m/n$.\n", + "\n", + "The discussion so far confirms that {eq}`implex` holds when $t$ is rational.\n", + "\n", + "So now take any $t \\geq 0$ and rational sequences $(a_n)$ and $(b_n)$\n", + "converging to $t$ with $a_n \\leq t \\leq b_n$ for all $n$.\n", + "\n", + "By property 1 we have $f(a_n) \\leq f(t) \\leq f(b_n)$ for all $n$, so\n", + "\n", + "$$\n", + " f(1)^{a_n} \\leq f(t) \\leq f(1)^{b_n}\n", + " \\quad \\forall \\, n \\in \\NN\n", + "$$\n", + "\n", + "Taking the limit in $n$ completes the proof.\n", + "```\n", + "\n", + "(fail_mem)=\n", + "### Failure of Memorylessness\n", + "\n", + "We know from the proceeding section that any distribution on $\\RR_+$ other\n", + "than the exponential distribution fails to be memoryless.\n", + "\n", + "Here's an example that helps to clarify (although the support of the distribution is a proper subset of $\\RR_+$).\n", + "\n", + "A random variable $Y$ has the Pareto distribution with positive parameters $t_0, \\alpha$ if\n", + "\n", + "$$\n", + " f(t) \n", + " := \\PP\\{Y > t\\} \n", + " = \n", + " \\begin{cases}\n", + " 1 & \\text{ if } t \\leq t_0\n", + " \\\\\n", + " (t_0 / t)^\\alpha & \\text{ if } t > t_0\n", + " \\end{cases}\n", + "$$\n", + "\n", + "As a result, with $s > t_0$,\n", + "\n", + "$$\n", + " \\PP \\{Y > s + t \\,|\\, Y > s \\}\n", + " =\n", + " \\frac{ \\PP \\{Y > s + t \\} }\n", + " {\\PP \\{Y > s\\}}\n", + " = \\left( \\frac{t}{t + s} \\right)^\\alpha\n", + "$$\n", + "\n", + "Since this probability falls with $s$, the distribution is not memoryless.\n", + "\n", + "If we have waited many hours for an event (i.e., $s$ is large), then the\n", + "probability of waiting another hour is relatively small.\n", + "\n", + "\n", + "## Sums of Exponentials\n", + "\n", + "A random variable $W$ on $\\RR_+$ is said to have the [Erlang\n", + "distribution](https://en.wikipedia.org/wiki/Erlang_distribution) if its\n", + "density has the form\n", + "\n", + "$$ \n", + " f(t) = \\frac{\\lambda^n t^{n-1}}{(n-1)!} e^{-\\lambda t}\n", + " \\qquad (t \\geq 0)\n", + "$$\n", + "\n", + "for some $n \\in \\NN$ and $\\lambda > 0$.\n", + "\n", + "The parameters $n$ and $\\lambda$ are called the **shape** and **rate**\n", + "parameters respectively.\n", + "\n", + "The next figure shows the shape for two parameterizations." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "tags": [ + "hide-input" + ] + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXwU9f348ddnNxchCSEh4UogCYT7vgQRRPEApCJorVivaotWrW1t9Wdba7W11fZrra1arbVWrQe1HoCKB6LIISLhkCSEQAiBXOSEJCTk3M/vj9lgCAnZJLs7s5v38/HIYzazM/N5D8d7Z9/zmc9Haa0RQgjhv2xmByCEEMKzJNELIYSfk0QvhBB+ThK9EEL4OUn0Qgjh5wLMDqAt/fr10wkJCWaHIYQQPmPHjh2lWuuYtt6zZKJPSEggJSXF7DCEEMJnKKUOt/eelG6EEMLPSaIXQgg/J4leCCH8nEs1eqXUAuCvgB14Xmv9aKv3RwH/BqYAv9JaP+bqvkII72poaCAvL4/a2lqzQxFdEBISQlxcHIGBgS7v02GiV0rZgaeBi4E8YLtSao3Wem+LzcqBu4ArurCvEMKL8vLyCA8PJyEhAaWU2eGITtBaU1ZWRl5eHomJiS7v50rpZgaQpbXO1lrXAyuBJa0aL9ZabwcaOruvEMK7amtriY6OliTvg5RSREdHd/rbmCuJfjCQ2+L3POc6V7i8r1JqhVIqRSmVUlJS4uLhhRBdIUned3Xl786VRN/WUV0d29jlfbXWz2mtp2mtp8XEtNnnX3RXYz3kbIEvn4Wqo2ZHI4TwElduxuYB8S1+jwMKXDx+d/YV7nKiGN6/Gw5+BvUnjHUb/wRXPAsjLjE3NiGEx7lyRb8dSFZKJSqlgoBrgDUuHr87+wp3cDTBW7fAgXUw4TvwnVfgB59C+CB47dvw4S+MK30hhN/qMNFrrRuBO4GPgAzgDa11ulLqNqXUbQBKqQFKqTzgbuB+pVSeUiqivX09dTKiDZseh0MbYdH/weLHYfS3YPBU+P4ncM5t8OXf4dPfmh2lEF128803Exsby7hx405b/+GHHzJy5EiGDx/Oo4923Ks7NTWVoUOH8swzz3QrHlfbTUhIYPz48UyaNIlp06Z1q80Oaa0t9zN16lQt3ODQZq0fjNT6zVu0djja3mb1j7R+sK/WhanejU2YZu/evWaH4Faff/653rFjhx47duypdY2NjTopKUkfPHhQ19XV6QkTJuj09PQOj/XFF1/omTNndjmWzrQ7dOhQXVJS0qV22vo7BFJ0OzlVnoz1VzXlRsmmbwIs/gu0d6f+ogehVyS891NwOLwYoOjpli5dyv3338+cOXMYMGAAn3zySZeOM3fuXKKiok5b99VXXzF8+HCSkpIICgrimmuuYfXq1R0eKzY2lvT0rhcdutqup1ly9ErhBl89B1WFsOJzCA5vf7vQKLjk97DqNtj1Mky9yWshCvM99G46ewsq3XrMMYMi+M23xna4XVpaGrNnz2bTpk28/fbbvPrqq1x00UWn3p8zZw5VVVVn7PfYY4+dtl1b8vPziY//ph9IXFwc27Zt6zCm++67j7q6Og4fPszQoUNPe8+VeDrTrlKKSy65BKUUt956KytWrOgwvq6SRO+Pmhphx0swbD4MmtTx9hOvgV2vwLrfwMjLIEy6twrPqqmpoaKigp/+9KcANDY2EhkZedo2mzZt6vLxjUrG6Trqf/7hhx9SXV3NZZddRnp6+hmJ3pV4OtPuli1bGDRoEMXFxVx88cWMGjWKuXPndthGV0ii90f7P4SqAuMGrCuUMso7z8yCzY/Dgkc8G5+wDFeuvD0hPT2dqVOnYrfbAdizZ88ZN1O7c0UfFxdHbu43z2rm5eUxaNCgdrevra3l3nvvZc2aNfz73/8mLS2NRYsWdTqezrTbvD42NpalS5fy1VdfSaIXnZDygtF9csQC1/eJGQFjlsCuV+HC+yGot+fiEz1eWloakyZ9821zz549LFly+ugo3bminz59OgcOHODQoUMMHjyYlStX8tprr516f/78+bz88ssMHmw8qP/www9zww03nOoJs2bNmb3AXYmno3abVVdX43A4CA8Pp7q6mo8//pgHHnigy+fbEbkZ62/Ks+Hgeph6I9g7+Tk+YwXUVcCeNzwTmxBOqamppyX6tLS0M67oXbV8+XJmzZpFZmYmcXFx/Otf/yIgIICnnnqKSy+9lNGjR3P11Vczdqzx7cXhcJCVlXXqBm5mZibr1q3jJz/5CQDjx48nLS2tS7GcrV2ARYsWUVBQQFFREeeddx4TJ05kxowZXHbZZSxY0IkLs05SbdWUzDZt2jQtUwl20boH4Iun4KdpENH+V9U2aQ3/mGMsb9vcfk8d4dMyMjIYPXq02WGYJi0tjRdeeIHHH3/c7FC6rK2/Q6XUDq11mx3y5YrenzTWGTdVRy7sfJIHI7FP/wEUpcGRre6PTwgLGDdunE8n+a6QRO9P9r0HNWUw7eauH2P8tyGkj9E9UwjhFyTR+5P9H0FoNCTN6/oxgkJh8vWQ8S5UFrorMiGEiSTR+wuHA7LWG33nbfbuHWvazcZgaDtfdk9sQghTSaL3F4W7oaYUhp+9f7FLoodBwnmQ9pZxY1YI4dMk0fuLrE8ABcPnu+d4Y5ZAaSYUZ7jneEII00ii9xdZn8CgydC7n3uON/pyUDbYu8o9xxNCmEYSvT+oKYe87e4p2zQL7w9DZ0P6O1K+EcLHSaL3B9mfgXZA8sXuPe7YK6B0PxTvde9xhRBeJYneH2Sth5BIY+Yod2ou36RL+UZYl6/NMJWZmcmkSZNO/URERPDEE08AHpx1qr0ZScz8kRmmOqGpSev/S9b6jZs8c/wXF2v9t6ntz1AlfI7MMNU+b84w1bx9//79dU5Ojtba9VmnZIapnqYoDU4Uub9s02zMFVB2AIpkql/hXjLDFKxfv55hw4adMfa9u8kwxb7u4KfGcpibulW2NvpyWPtz46bsgK6NLigs7IP74Giqe485YDws7LhU0tNnmAJYuXIly5cvP/W7p2adkkTv63K/gujhRi8ZTwiLgSHnGsMrzP+1Z9oQPY7MMAX19fWsWbOGRx75ZqIfT806JYnel2kNeV+5t1tlW5Ivhk9+A5UFXRsVU1iXC1feniAzTMEHH3zAlClT6N//m4s0T806JYnelx3LgeoSiJvu2XaSLzES/YF1xoQmQnRTT59hCuD1118/rWzjyVmn5GasL8vbbizjZ3i2ndjREBEHBz72bDuix+jpM0zV1NSwbt06li1bduo9T846JTNM+bL3fw5fvw73Hen+iJUdee+nxhSD9x6CgCDPtiU8SmaYkhmmhC/J2w6Dp3g+yYNRvqk/AUe+8HxbQniQzDAlfEd9jdGH3tP1+WaJc8EeZNTphRA+RRK9ryrYBY5GiPNwfb5ZUG9jjHqp0wvhcyTR+6q8r4ylt67owSjflO6H8kPea1N4hBXvzQnXdOXvThK9r8rdDlHDoHe099pMvsRYZnXtUXVhDSEhIZSVlUmy90Faa8rKyggJCenUftKP3hdp7Rx/3kPDHrQnehhEJRl1+hk/8G7bwm3i4uLIy8ujpKTE7FBEF4SEhBAXF9epfVxK9EqpBcBfATvwvNb60VbvK+f7i4Aa4Cat9U7nez8Fvg9oIBX4nta6tlNRitMdPwzVxRDnxmFMXTXsQtj9OjQ1gD3Q++2LbgsMDCQxMdHsMIQXdVi6UUrZgaeBhcAYYLlSakyrzRYCyc6fFcAzzn0HA3cB07TW4zA+KK5xW/Q9Va7zQSlv3YhtKWkeNFRDnjznIISvcKVGPwPI0lpna63rgZXAklbbLAFedg6L/CUQqZQa6HwvAOillAoAQoECN8Xec+WnQGAoxLb+vPWChPOMyUiyN3i/bSFEl7iS6AcDuS1+z3Ou63AbrXU+8BhwBCgEKrTWbfbPU0qtUEqlKKVSpHbYgaOp0H8c2E24xdKrLwycBIc+937bQogucSXRtzXGZuvb9W1uo5Tqi3G1nwgMAnorpa5rqxGt9XNa62la62kxMTEuhNVDaW0k+gHjzYshaZ5xM7juzJH8hBDW40qizwPiW/wex5nll/a2uQg4pLUu0Vo3AG8D53Y9XMHxw1BXaX6idzTCYRkOQQhf4Eqi3w4kK6USlVJBGDdTW4/huQa4QRlmYpRoCjFKNjOVUqHOnjnzgQw3xt/zNM8GNGCCeTHEnwMBIVKnF8JHdFjk1Vo3KqXuBD7C6DXzgtY6XSl1m/P9Z4G1GF0rszC6V37P+d42pdSbwE6gEdgFPOeJE+kxjqYaN0NjTRx9MDAEhsyURC+Ej3Dpbp7Wei1GMm+57tkWrzVwRzv7/gb4TTdiFC0dTYXoZAgKNTeOpHnwyYNQVeS5aQyFEG4hQyD4GrNvxDZLmmcspfeNEJYnid6X1JRDRa41Ev2ACUZXSynfCGF5kuh9SZFzejMrJHqbHRLmwKGNRpdPIYRlSaL3Jad63Fgg0YMxGUlFrjFJuRDCsiTR+5KjqRA2AMJizY7EkHCesczZbG4cQoizkkTvS6xyI7ZZzCjoHQM5m8yORAhxFpLofUVjPZRkWivRK2Vc1R/aJHV6ISxMEr2vKNkHjgZrJXowEn1VAZRnmx2JEKIdkuh9hRWGPmhLwlxjKeUbISxLEr2vOJpqjEEfZbGZgfolQ1h/o3wjhLAkSfS+ojjdGN/GZjc7ktM11+lzNkudXgiLkkTvK4r3QYyJA5mdTcIcOHEUyrLMjkQI0QZJ9L6gptyYDDx2lNmRtC1hjrGUOr0QliSJ3hcUO4fwt+oVffQwCB8odXohLEoSvS8o3mssrXpFr5RxVS91eiEsSRK9LyjZB8ERENF6TnYLSZhtlJdKD5gdiRCiFUn0vqB4nzHcgGprDnaLaK7TH5Zxb4SwGkn0vqAkw7plm2ZRScaAazLAmRCWI4ne6k6UQE2ZdW/ENjvVn36L1OmFsBhJ9FZX4uxxY/UrejDq9CeOQtlBsyMRQrQgid7qivcZS6tf0YPU6YWwqACzAxAdKMmAkD4QPsDsSDoWPRx6xxp1+qk3tbuZ1pqv8yrYklVKZW0DJ+ubaHJoJsZFMmtYNPFRod6LWYgeQBK91TUPfWDlHjfNWtfpW8VcXl3PS1/ksHp3PjllNQAEBdgIDbLjcGhe3XYEgPioXtw8O5FrzxlCcIDFxvYRwgdJorcyrY0r+jFLzI7EdQmzIf1tY3z66GGnVn+cfpRfvpNKWXU9s5KiuX3ecC4dN4A+vQIB4yr/QPEJth4s4/3UQh56dy/PbzrETy5K5sopcdhsPvBBJ4RFSaK3shNFcPKYb9Tnm52q02+B6GFU1jbw4Jp03t6Zz5iBEfznlnMYPTDijN2UUozoH86I/uHcMGsom7NK+dOHmdzz5h7e21PIX6+ZRGRokJdPRgj/IDdjrazYh3rcNOs3wjmP7GYqTjbw3X9uY/XuAu66cDir7pjdZpJvTSnFnOQY1tw5m4evGMfWg2UsfnIzafkVXjgBIfyPJHorK/GhHjfNlIKh5+I4tIkbnv+SfUcree76qdx9yUiCAjr3z00pxXUzh/LGbbNocmiWPfMFH6QWeihwIfyXJHorK86AXlEQFmt2JJ1ycvC52KoKqDqaxTPfncr80f27dbxJ8ZG896PzGD+4D3e+vkuSvRCdJIneykoyrT/GTSsOh+a3qVEAPDmrhovGdC/JN4sOC+alm2cwKT6SH72+iw/TjrrluEL0BJLorax0P8SMMDuKTnnm84O8nhNKbWBfxtbvceuxw4IDePF705kQ14c7X9vJp/uK3Hp8IfyVJHqrqi6Fk+XQb6TZkbhsW3YZf/44k8snDiY4ea5HxqcPDwnkpZtnMGZQBD96bRf7jla69fhC+COXEr1SaoFSKlMplaWUuq+N95VS6m/O9/copaa0eC9SKfWmUmqfUipDKTXLnSfgt0oyjWU/37iiLz1Rx10rdzE0ujd/WDYelTAHKvPg+GG3txUeEsg/b5hGWEgAt7yYQumJOre3IYQ/6TDRK6XswNPAQmAMsFwpNabVZguBZOfPCuCZFu/9FfhQaz0KmAhkuCFu/1fqTPQ+Urr5xdupHKtp4OlrpxAWHGA8IQseG7a4f0QI/7xhGmXVddz2nx3UNTZ5pB0h/IErV/QzgCytdbbWuh5YCbR+VHMJ8LI2fAlEKqUGKqUigLnAvwC01vVa6+NujN9/lR6AwFCIiDM7kg59sreIdXuL+OlFIxgzyNlPPmYUhEZ7dHz6CXGRPPbtiaQcPsZD7+71WDtC+DpXEv1gILfF73nOda5skwSUAP9WSu1SSj2vlOrdViNKqRVKqRSlVEpJSYnLJ+C3SjKhXzLYrH0b5WR9Ew++m05ybBi3nJf4zRunxr3x7DyyiycM4rbzh/HatiO8v0e6XQrRFleySFt9+1r/z21vmwBgCvCM1noyUA2cUeMH0Fo/p7WeprWeFhMT40JYfq50v0/U5/++IYu8Yyf53RXjznwgKmEOVOR6pE7f0s8uGcGk+Ejue3sPueU1Hm1LCF/kSqLPA+Jb/B4HFLi4TR6Qp7Xe5lz/JkbiF2dTX20kSIv3uDlYcoJ/fJ7N0smDmZkUfeYGQ2cbSw9PLxhot/Hk8smg4ccrd9HQ5PBoe0L4GlcS/XYgWSmVqJQKAq4B1rTaZg1wg7P3zUygQmtdqLU+CuQqpZoz1nxAiqkdKT1gLC1+I/Z37+0lONDGLxa1MxaPF+r0zeKjQvnDsvHsPHKcv60/4PH2hPAlHY5eqbVuVErdCXwE2IEXtNbpSqnbnO8/C6wFFgFZQA3wvRaH+BHwqvNDIrvVe6ItpfuNpYVLN9tzytmQWcJ9C0cRGx7S9kY2m3FVf2hTm+PTu9u3Jg7i8/0l/H3DQS4e058JcZEebU8IX+HSMMVa67UYybzlumdbvNbAHe3suxuY1o0Ye56STFB2iBrW8bYm0Frz2EeZ9AsL5sZZCWffOHEuZKyBYzkQlXj2bd3g14vHsOlACff8bw9rfjRbJi4RAnky1ppK9xtJMcCa469vySpj26Fy7rxgGL2COkikzePTH9ro+cCAPr0CeWTZeDKLqnjq0yyvtCmE1Umit6LS/Za9Eau15rGPMxnUJ4Tl5wzpeIeYkcY8sl5K9AAXjurPlVPi+PuGgzKGvRBIoreepkYoO2j0obeg9RnF7M49zl3zk10riyhllG9yNnm0P31rDyweQ3TvIO55cw+N0gtH9HCS6K3mWA44GowrYYvRWvPE+v0MjQ7lyqmdeGI3ca4xLWLzTWYv6BMayG+XjCWjsJIXv8jxWrtCWJEkeqtpHuPGgqWbrQfLSMuv5IfnDyPQ3ol/OonerdM3u3TsAC4cFcvj6/ZTcPykV9sWwkok0VvNqVErrVe6+cfGbPqFBXPF5NYjYHSgbyL0ifd6oldK8dDlY3FozYNr0r3athBWIoneakr3Q/hACOl4Em1vyjxaxef7S7jp3KGEBHayy2LLOr3Du/Xy+KhQfjx/BB87B14ToieSRG81Fh3j5p+bsukVaOe75wzt2gES58LJY1CU5t7AXPD9OYmM7B/Og2vSOVkvwxmLnkcSvZVobQx/YLEbsUcralm9O5/vTI+nb+8u9u1v7k+fs8l9gbko0G7joSVjyT9+kmc/P+j19oUwmyR6K6k6CnWVlruif/GLHJoc+vRhiDurz2DjSV8v1+mbzUyK5lsTB/Hs5wdlhEvR40iit5JTY9xY50bsyfomXtt2mIXjBhIfFdq9gyXOhZwtxrMCJvjlolHYlOLh92VcPdGzSKK3klOJ3jqlm3e/LqCytpEbz03o/sGSzof6KijY2f1jdcHAPr2488LhfJRexKYDMrmN6Dkk0VtJ6X4ICofwAWZHcsor2w4zon8Y0xP6dv9giecDCrI3dP9YXfT9OYkMjQ7lwTXpMm696DEk0VtJSaYxBr2Hh/N11de5x9mTV8F1M4ei3BFTaBQMnAgHP+v+sbooOMDOry8bw8GSal790rMzXwlhFZLoraT0gKVuxL7y5WFCg+ws7ewDUmcz7ALI+wrqqtx3zE6aPzqW84b34y+fHOB4Tb1pcQjhLZLoraKuCqoKLHMj9nhNPWu+LuCKyYMJDwl034GT5oGj0bgpaxKlFPcvHk1VbQNPfCKzUQn/J4neKix2I/bNHXnUNTq4rqsPSLUnfiYEhEC2eeUbgFEDIlg+Ywj/+fIwWcXmfbsQwhsk0VtF8zyxFijdaK15ddsRpg7ty5hBbh6KITAEhp5rap2+2d0XjyA0yM7D72eYHYoQHiWJ3ipKMsEW4JXp9jry1aFyDpVWs3yGCxOLdEXSBcYonZUFnjm+i6LDgrnrwmQ2ZJbw+X7pbin8lyR6qyjdD1FJYHdjPbyL3kjJIyw4gEXjPdTNM2mesTSxm2WzG84dypCoUP7wfgZNDu9NjCKEN0mitwqL9Lipqm1gbWoh35o4iNAgl+aO77z+4yC0nyXKN8EBdu5bOIrMoireSMk1OxwhPEISvRU0NUD5QUsk+vf2FHKyoYmrp3ViBqnOstmMq/rsDV6dXrA9C8cNYNrQvvz540xO1JkzPIMQniSJ3gqO5RhdDi2Q6N9IySU5NoxJ8ZGebWjYBVBdbMqwxa0Z3S3HUHqinmc3yOiWwv9IoreC5lmlYsxN9AeKqth15DhXT4t3z5OwZzNsvrPRdZ5tx0WT4iNZMmkQ/9yUTb5MOyj8jCR6K2juQx9t7sNSb6TkEmBTLJ3ixidh2xMxEPqPh6xPPN+Wi+65dCQaeOyjTLNDEcKtJNFbgQWmD2xocvDOrnzmj46lX1iwdxpNvgiOfAm1Fd5prwNxfUO5eXYi7+zKJzXPGjEJ4Q6S6K3AAtMHbtxfQumJeq6aGu+9RodfDLrJEt0sm91+wTCiegfx8Pt70Ra4USyEO0iiN5vWUGJ+on97Zz5RvYM4f0SM9xqNnwHBEZap0wNEhATy04uS2XaoXCYTF35DEr3ZKguMyThiR5kWQsXJBtZlFPGtCQMJCvDiPwl7oNHNMmu9JbpZNrtmxhCSYnrz6Af7ZMx64Rck0ZutZJ+xjDEv0a9NLaS+0cGyKR7sO9+e5IuNUTuL0r3fdjsC7TZ+uXA02aXVvLbtiNnhCNFtkujNZoFE/87OfJJiejMhro/3Gx9+sbE88LH32z6L+aNjOXdYNE98sp+Kkw1mhyNEt7iU6JVSC5RSmUqpLKXUfW28r5RSf3O+v0cpNaXV+3al1C6l1HvuCtxvlOyD0Gjo3c+U5o+U1fBVTjlXTonzfN/5tliwmyUYD1H96rLRHD/ZwNOfZZkdjhDd0mGiV0rZgaeBhcAYYLlSakyrzRYCyc6fFcAzrd7/MSBjwbalJNPcq/ld+QAsmTTItBis1s2y2dhBfbhqShwvbskht7zG7HCE6DJXruhnAFla62ytdT2wEljSapslwMva8CUQqZQaCKCUigMuA553Y9z+QWvjit6kRK+15p1decxMiiKub6gpMQCQfInRzTJrvXkxtOPnl47EblM8+uE+s0MRostcSfSDgZbD+uU517m6zRPAvcBZuy8opVYopVKUUiklJT1kbPATRcZVrEmJflfucXLKalg22YSbsC3FzYBeUZD5gblxtKF/RAi3np/E+3sK2XG43OxwhOgSVxJ9W4Xb1n3h2txGKbUYKNZa7+ioEa31c1rraVrraTExXuzLbaZTN2LNmT5w9a58ggJsLPDUuPOusgfAiAVw4CNjJE+LWTE3if4Rwfz2vQwcMma98EGuJPo8oOXjknFA66mB2ttmNnC5UioHo+RzoVLqlS5H629ODWbm/Sv6hiYH7+0p5KLRsUS4c/Lvrhq1yPh2c/gLsyM5Q2hQAPdeOoqvc4+z+ut8s8MRotNcSfTbgWSlVKJSKgi4BljTaps1wA3O3jczgQqtdaHW+hda6zitdYJzv0+11te58wR8Wsk+CImEsFivN705q5Sy6nqWTPLCAGauGHYh2IMtWb4BWDp5MBPi+vDHDzKpqZcx64Vv6TDRa60bgTuBjzB6zryhtU5XSt2mlLrNudlaIBvIAv4J3O6heP1Lc48bE7o1rt6VT0RIAPNGWqRMFtTbeEo2831LPSXbzGZT/HrxGI5W1vLcxmyzwxGiU1yaK05rvRYjmbdc92yL1xq4o4NjbAA2dDpCf1ayD0Yt9nqzNfWNfLy3iCWTBhEcYPd6++0atcio0xelw4BxZkdzhukJUVw2YSDPfn6Q70yPZ2CfXmaHJIRL5MlYs1SXQk2ZKfX5dXuLqKlvsk7ZptmIhYCybPkG4L4Fo3BoePQD6W4pfIckerMUO58fM6HHzapd+QzqE8KMhCivt31W4f0hbppRvrGo+KhQbp2bxOrdBaTkSHdL4Rsk0ZvFpDFuyk7UsfFAKZdPGozNZsKQBx0ZuQgKdhmjelrUD+cNY2CfEB58N50m6W4pfIAkerOUZEJQOER4d+iB91MLaXJoc4c8OJvmexYZ75obx1mEBgVw38JRpOVX8r+U3I53EMJkkujNUrLPKNt4ucfN6t0FjOwfzuiB5k1beFYxIyB2DKSvMjuSs7p84iCmDe3L/32UKaNbCsuTRG+W5kTvRbnlNew4fIwlky16Nd9s7FI4stXS5RulFA9ePpbymnqe+GS/2eEIcVaS6M1wohiqS6D/WK82u3q38VTn5RMtnujHXAFo2Nv6uTxrGTe4D9fOGMLLWw+z72il2eEI0S5J9GZonk3Ji4lea82q3QVMT+hr7kiVrogZAf3HQfo7ZkfSoXsuHUlESAAPrEqXycSFZUmiN8OpRO+9h4L2FlaSVXzCen3n2zP2Csj9EiqsPbZMZGgQ9y4YxVc55azabe1YRc8lid4MRekQ1t+rs0qt3l1AgE2xaPxAr7XZLWOWGsu9q82NwwXfmRbPxLg+/GHtPqpq5cassB5J9GYoSvNq2cbh0KzZXcD5I2KI6h3ktXa7pd9wGDDeJ8o3Npvit0vGUXqijj9/LDdmhfVIove2pkajx40XE/22Q+Ucrazlcqv2nW/P2KWQ9xUct35f9YnxkVx3zlBe3ppDap61pkQUQhK9t5VlQVO9V+vzq3bl0zvIziVjTJ5gpNGqDJ4AABlgSURBVLPGOss3aW+ZG4eL7lkwkuiwYH75Tqo8MSssRRK9txWlGUsvJfrahibWphayYNxAegVZaKRKV0QlGdMMfv26JYcubi0iJJAHFo8hNb+Cl7fmmB2OEKdIove2onSwBUC/EV5p7tN9xVTVNbJ0so/0tmlt0nKj1FW42+xIXLJ4wkDOHxHDYx9lUlhx0uxwhAAk0XtfUTr0GwkB3rkp+s6ufGLDg5k1LNor7bnd2KXGzFO7Xzc7EpcopfjdknE0ac2vpW+9sAhJ9N5WlO61G7HHquvZkFnMkkmDsFtxpEpX9OoLIxdC6v+gsd7saFwyJDqUuy8ewScZRby3p9DscISQRO9VJ49BZZ7XEv37qYU0NGmu8NWyTbNJ18LJcjjwsdmRuOzm2YlMjOvDg2vSKa/2jQ8o4b8k0XtT0V5j6aUbsat25TOifxhjrDpSpauGzYfescZNWR8RYLfxx6smUFnbwO/e22t2OKKHk0TvTV4c4+ZIWQ0ph4+xdHIcyoTJx93KHgATrob9H0GN78zqNGpABLfPG847u/L5dF+R2eGIHkwSvTcVpUGvKAj3fH/2t3floRTWnWCksyZeA44G2POG2ZF0yh0XDGdk/3DueyuV4zVSwhHmkETvTc03Yj18he1waN7amcfsYf0YFNnLo215zYDxMGgKpLzgE33qmwUF2Pjz1RMpr67ngdXpZocjeihJ9N7iaILivV4p22zPKSe3/CRXTvXxm7CtTb8FSjMhZ7PZkXTKuMF9uGt+Mmu+LuB96YUjTCCJ3ltK90NDDQyc5PGm3tyRR1hwAJeO9bEhDzoydhmERELKv8yOpNNunzeMiXF9uH9VKsVVtWaHI3oYSfTekr/TWA6e4tFmauobWZtayKLxAwgNCvBoW14XFAqTrzMmDq86anY0nRJgN0o41fVN/L8398iDVMKrJNF7S8FOCAqH6GSPNvNh2lGq65u4amq8R9sxzbSbwdEIO/9jdiSdNjw2nF8tGs1nmSW8+EWO2eGIHkQSvbfk74BBk8Dm2T/yN3fkMSQqlOkJfT3ajmmih0HSBbDj38aQzz7mhllDmT8qlkfW7mNvgcwzK7xDEr03NNbB0TSPl23yjtWwNbuMK6f4Qd/5s5l+C1Tmw/4PzY6k05RS/OmqCUSGBvKj13dysr7J7JBEDyCJ3huK0ow+4IM8m+jf2pGP1rBsip/1tmltxEKIiIMv/252JF0SHRbM41dPIru0mofelS6XwvMk0XuDF27ENjk0/91+hDnJ/YiPCvVYO5ZgD4BZd8DhLZC73exouuS85H788PxhrNyey1s78swOR/g5SfTeULALQvtBH8/dIN14oISCilqWzxjisTYsZcoNRlfLLU+YHUmX3X3xCGYmRfGrVansOyr1euE5LiV6pdQCpVSmUipLKXVfG+8rpdTfnO/vUUpNca6PV0p9ppTKUEqlK6V+7O4T8An5O42reQ/WzV/fdoR+YUFcNLq/x9qwlOAwmLEC9r0PJb45IXeA3cbflk8mIiSQH76yk6raBrNDEn6qw0SvlLIDTwMLgTHAcqXUmFabLQSSnT8rgGec6xuBn2mtRwMzgTva2Ne/1VUZMyR5sD5fVFnL+n3FXDk1jqCAHvQlbcYKCAiGL/5mdiRdFhsewpPLJ3OkvIZ7/if964VnuJIVZgBZWutsrXU9sBJY0mqbJcDL2vAlEKmUGqi1LtRa7wTQWlcBGYCf3ylspfBrQHu0Pv+/lFyaHJprpveQsk2zsBjjAao9/4VK3x1a4JykaH6xcBQfph/lyU+zzA5H+CFXEv1gILfF73mcmaw73EYplQBMBra11YhSaoVSKkUplVJSUuJCWD6i+Uash67oHQ7Nyu25nDssmsR+vT3ShqXNutN4gGrrU2ZH0i23nJfIsimDeXzdfj5M862nfoX1uZLo2yost/5+edZtlFJhwFvAT7TWbd510lo/p7WeprWeFhMT40JYPqJgp3ETNswz57Q5q5S8Yye5pqfchG0tKhEmfAe2P+/TV/VKKf6wdDyT4iO5+43dcnNWuJUriT4PaNldJA4ocHUbpVQgRpJ/VWv9dtdD9VHNN2I95OWth4nuHcSlY3vITdi2zLvPGB1045/MjqRbQgLt/OP6qYSHBHDLiykUV8rgZ8I9XEn024FkpVSiUioIuAZY02qbNcANzt43M4EKrXWhMh7P/BeQobV+3K2R+4ITJXD8sMfKNofLqlm/r4hrzxlCcIDdI234hL4JMPVG2PkylGebHU239I8I4fkbpnOspp6bX9pOdZ3vDfMgrKfDRK+1bgTuBD7CuJn6htY6XSl1m1LqNudma4FsIAv4J3C7c/1s4HrgQqXUbufPInefhGXlbDKWQ2d75PAvbz2MXSmumznUI8f3KXPvAVsgfPaI2ZF02/i4Pjx97RT2FlRy52s7aWxymB2S8HEujWOrtV6Lkcxbrnu2xWsN3NHGfptpu37fM+RsgqAwYzAzN6uua+SN7bksGj+Q/hEhbj++zwkfAOfcClv+Cuf9xCsTvHjSBaNiefiK8fzynVTuX5XGI8vG+/f4RcKjelCnaxMc2gRDZoE90O2HfmtnHlV1jdw0O8Htx/ZZs38MwRGw7gGfmm6wPdeeM4Q7LxjOyu25PPrBPuljL7pMEr2nVBZC2QFInOP2Qzscmhe/yGFiXB8mx0e6/fg+KzQK5v0/yPoEMtd2vL0P+NklI7h+5lD+sTGbpz+TPvaiayTRe0rzvKaJc91+6E1ZpWSXVPO92Ynydb61GSsgZjR8cB/U15gdTbcppXjo8rEsmzyYxz7ez7+3HDI7JOGDJNF7Ss5GCOkDAya4/dD/+PwgMeHBLBo/0O3H9nn2QLjsMag4Apv/YnY0bmGzGWPYLxg7gIfe3cuLkuxFJ0mi95RDm4zeNjb3dnvceeQYXxwsY8WcpJ41rk1nJJwH4682RrYsO2h2NG7RPADapWP78+C7e3l+k293IxXeJZnCEyry4NghSHB/ff7pT7OIDA3k2nN66JOwrrrkd2APhvfv9osbswBBATaeunYKl40fyMPvZ0jNXrhMEr0nHHL2n3fzjdi9BZWs31fMzbMT6R3sUs/Ynit8AFz8EGRvMIZH8BOBdht/vWYSV0waxP99lMkf1mbgcPjHB5nwHMkWnpCzCXpFQax7+3I/vSGLsOAAbpyV4Nbj+q1pNxu9bz7+NSTNg37JZkfkFgF2G3++ehIRvQJ5bmM2JVV1/OmqCQTa5bpNtE3+Zbib1nBoIyTMBpv7/ngPlpxgbWoh188aSp9Q9/fL90tKweVPQWAIvHMrNPnPcAJ2m9Eb5+eXjOCdXfnc8lKKTFwi2iWJ3t3Ks6EiFxLc263yqU+zCA6wcct5iW49rt+LGAiL/wL5O2DTY2ZH41ZKKe68MJk/XjmeLVmlLPv7Fxwp8/0upcL9JNG7297VxnLkArcdMi2/glW787np3ET6hQW77bg9xtilMOEa2PCo8TCVn/nO9CH85+YZFFfVseTpzWw9WGZ2SMJiJNG7295VMHgqRLqnV4zWmkc+yCCyVyC3XzDMLcfskRY/box/8+bNPj/CZVvOHd6P1XfMJqp3ENf/axvPb8qWIRPEKZLo3ak825g6cOxStx1yw/4StmSVcdf8ZCJCpDbfZUG94ZpXQdlg5Xeh7oTZEbldQr/evHPHbC4cFcvD72dw2ys7qDgpdXshid690lcZyzGtp9TtmiaH5tG1+xgaHcp3z5GhiLutbwJc9W9jsvZVtxmTlfiZiJBA/nH9VO6/bDTrM4r51pOb2XnkmNlhCZNJoncnN5dt3tyRS2ZRFfdeOkqegnWXYRfAJb+HjHf96mGqlpRSfH9OEv+9dRZNDs1Vz3zBnz/OpL5RxrXvqSR7uEv5IaNsM+YKtxyu7EQdf/wwkylDIlk0foBbjimcZt0O590NO16ETx40OxqPmTq0Lx/8ZA7LpsTx5KdZLHtmCxmFMhdtTySJ3l32urds89v39lJV28AjyybICJWeMP8BmHaLMR7OJv+d5TIiJJDHvj2RZ6+bSuHxWhY/uZlHPsjgZL3/la1E++TJWHdJX2XMDdu3+7X09RlFrN5dwE8uSmbkgHA3BCfOoBQsegzqKmH9Q1BfDRfeb6z3QwvGDeCcxCge+SCDf3yezdrUQn6zeCzzR8fKhUQPIFf07lB2EAp3u6W3TWVtA796J42R/cO5fd5wNwQn2mWzwRXPwpQbjIep3v2xXz0921rf3kH86aqJrFwxk+AAO99/OYXr/rVNyjk9gCR6d9j6NNiDYPy3u32o37+XQXFVLX+8aoLcgPUGewB8628w52ew8yX4341+2fWypZlJ0Xzw4zk8dPlY0gsquexvm7jnf1+TWy5P1forySTdVVUEu16BicuNx+274X8pufw3JZdbzx/GJJki0HuUMmr2Cx41BkH754VQst/sqDwq0G7jxnMT2PDzeXxvdiKrvy7gwj9v4FfvpFJw/KTZ4Qk3k0TfXdueAUeDMTF1N6TlV/CrVWnMHh7Nzy4e4abgRKfM/CFc/w7UlME/L4C0t82OyOMiQ4P49eIxbLznAr4zPZ43UnKZ+6fPuPuN3WQerTI7POEmyoqPSU+bNk2npKSYHUbHaivgL+Ng+Hz49otdPkx5dT3fetKYY3bNnbOJlvFszFWRD/+7CfK+MspxC/4IvaPNjsor8o+f5PlN2az8KpeTDU3MHRHDDTOHcsGoWOw2uWlrZUqpHVrraW2+J4m+GzY9bvTYuHUjDJzYpUPUNjTxvX9vZ8eRY7x52ywmxEnJxhIa62Hz47DxMQiJgIV/gnFX+m2vnNaOVdfzny8P8+q2wxRV1jE4shfLZ8SzbEocgyJ7mR2eaIMkek9oOAlPjDcm/76+a1/xaxua+MHLKWzOKuUvV0/iismD3Ryk6LaidFh9JxTshCGz4OLfQfx0s6PymoYmB+v2FvGfrYfZml2GUjB7WD+WTh7MxWP7y/hLFiKJ3hPWPQBb/go3rTUmGemk2oYmVvxnB5sOlPCnKyfw7WnxHghSuEVTo9EjZ8OjUF0Moy+H8++FAePNjsyrjpTV8NbOPN7elUdu+UmC7DbmJPdj4fiBXDAyRkqOJpNE725Z6+GVZcZUdYv/0undj9fUc+dru9hysJQ/LpvA1dMlyfuEuhOw9Sn44kmoPwFJF8Dsu4xlDynpADgcmt15x1m7p5C1qYUUVNSiFEwZ0pcLRsZwXnIM4wf3kZq+l0mid6eqInh2NvSOgR98CoGdq1d+nXuc21/dSXFVLX9YOl6u5H3RyWOQ8gJs+wecKIKoJJh0rdHFtk+c2dF5lcOhSS+oZP2+ItZnFJOaXwFAREgAM5OimZEYxfSEKMYMipA5bT1MEr27OBzGlfyRL2HFZxA72uVdmxya/2zN4Q9r9xETHszfvzuFidJX3rc11hldMHe9Aoc3AwqGzIRRi2HUZRDV86Z9LD1RxxcHy9hyoJQvskvJLTf65PcKtDNucAQT4iKZENeHsYMiSOwXJlf9biSJ3h3qq2HNXZD2Jix+AqZ9z+VdNx0o4ffvZ7DvaBUXjIzh8asn0bd3kAeDFV5Xfgj2/Bcy3oOiVGNd1DBIOh8S5xo3csN73iikRZW1pOQcI+VwOXvyKkjLr6DOOVxySKCNkf3DSe4fTnJsGMNjw0js15v4qFC5+u8CSfTdVXYQ/nsdFGcYA1/N+VmHNdnGJgcbMkt4aWsOmw6UEh/Vi/+3YBSXjR8og0j5u2M5kPkBZG+AnC1Q73zwKGKwMV/BwAkQO9b4Rhg51Bhzp4doaHJwoOgEewsrySisZN/RSg4UnaC4qu7UNnabYnBkL4ZEhRIf1Yu4vqEMigxhYJ9eDOwTQv+IEEIC7SaehTV1O9ErpRYAfwXswPNa60dbva+c7y8CaoCbtNY7Xdm3LZZJ9FVF8PVrsOkvxn/GK5+H4Re1u3lNfSM7Dh9j4/4SVu0uoKSqjn5hwayYm8iN5yYQHCD/OHucpkZjwLvcryB/B+SnGB8EzezBRoknKsmYAatPnPGBEDEYwmKgdywEhZoVvddUnGwgq/gEh0qrOVxWzaHSanKPnST/WA2lJ+rP2D48JIDY8GD6hRk/0WFB9A0Nom9oIH17BxEZGkSfXoFEhAQQ0SuQ8JAAv///161Er5SyA/uBi4E8YDuwXGu9t8U2i4AfYST6c4C/aq3PcWXftpiS6B0Oo+tcSSaU7jeuxvZ/CI5GSJgDS56mqc8QqusbOVZdT1l1PcWVdeSUVXOopJr9xVWk5lXQ6NAE2BTzRsbynenxzBsZI19DxelqK41/Z8XpUJYFZdlQfhCOH4GGNgYWCwqDXlEQGgW9+hoPcIX0geAI473gMGNO3MBQo3NAYCgEBENAiDHYXkCwsbQHGktboDGYmy0QbHawBRhz6Vr0m2ZNfSOFFbUcrail4PhJiqvqKK6spbiqjrIT9ZRWG8uO5scNstsICwkgNMhOWLCxDA0KoFeQnV6Bxk9IoI2QIDvBAcbr4AA7QQE2gu02ggKMn0C7jQC7IshuI8CmCAywEWizYbcpAu0Ku00RYLNhtyvsSmGzYfzufG23KWzK+DFe45Zv+WdL9K6MRz8DyNJaZzsPthJYArRM1kuAl7XxqfGlUipSKTUQSHBhX7c58LupBOq6M9a3/iNUOLDhwK4dBFJPqD5JKLWnbVNGJB/YF7Mq4EKyjgzk5J/3UteY1ma7/cKCSIoJ4wdzk5iZFM3UoX0JC5ah/kU7QiKMh65aP3iltdGjpzIfKguNC48TRVBdCjXlcLLcWFbmGx8WtRXQ6MYByJQNlN1I/qde24zXqBYfBq1ft17Cqf91zetOvT7VWIuXZ18fCgxz/rQRNAQDwaAxOj04tG61BIfWOJpfN2kc1Rp9wrleg9YarcGBsWzv+rej+ocGGp0/XVFt78Pk32zt4t7tcyUbDQZyW/yeh3HV3tE2g13cFwCl1ApgBcCQIV2bc7WidwI2x5lf8wA0p/9jcmBDKzsNKpBaWyh1tlBO2CMpDh5CcfBQqgJjCLDbSLYrxthtxid+kJ3QIDtRvYOJ7h1Ev7BghkSH0qeXPB0o3EAp46o9NMr1h7EcTUZHgfpq49tAY63x1HZjrfN1LTTVQ1ODsXQ0GOUkR4PxbdXRaPyum4xjnVo6Tv9xNAHNGVA71ze/ptV7zemw1etmp2VRF9a3p1U2VrhvJiUNzg+K5g8DjcPhMD4M9DcfHrrFUjd/aPDN79oZ5mmvMbZp/qPTzj8/rSEoKMJNZ3A6V/5c2vpO0fpvob1tXNnXWKn1c8BzYJRuXIjrDNPufqsruwnhu2x2ZynHMwmip1IYNxX9parvSqLPA1o+1RMHFLi4TZAL+wohhPAgV+4SbgeSlVKJSqkg4BpgTatt1gA3KMNMoEJrXejivkIIITyowyt6rXWjUupO4COMbzIvaK3TlVK3Od9/FliL0eMmC6N75ffOtq9HzkQIIUSb5IEpIYTwA2frXikdvIUQws9JohdCCD8niV4IIfycJHohhPBzlrwZq5QqAQ53cfd+QKkbw/EFcs7+r6edL8g5d9ZQrXVMW29YMtF3h1Iqpb07z/5Kztn/9bTzBTlnd5LSjRBC+DlJ9EII4ef8MdE/Z3YAJpBz9n897XxBztlt/K5GL4QQ4nT+eEUvhBCiBUn0Qgjh5/wm0SulFiilMpVSWUqp+8yOxxOUUi8opYqVUmkt1kUppdYppQ44l33NjNHdlFLxSqnPlFIZSql0pdSPnev99ryVUiFKqa+UUl87z/kh53q/PWcw5qdWSu1SSr3n/N2vzxdAKZWjlEpVSu1WSqU417n9vP0i0TsnIX8aWAiMAZYrpcaYG5VHvAgsaLXuPmC91joZWO/83Z80Aj/TWo8GZgJ3OP9u/fm864ALtdYTgUnAAuc8D/58zgA/BjJa/O7v59vsAq31pBb9591+3n6R6GkxgbnWuh5onoTcr2itNwLlrVYvAV5yvn4JuMKrQXmY1rpQa73T+boKIxEMxo/PWxtOOH8NdP5o/PiclVJxwGXA8y1W++35dsDt5+0vib69ycl7gv7O2bxwLmNNjsdjlFIJwGRgG35+3s4yxm6gGFintfb3c34CuBdwtFjnz+fbTAMfK6V2KKVWONe5/bzdNWm62VyehFz4JqVUGPAW8BOtdaVSbf2V+w+tdRMwSSkVCbyjlBpndkyeopRaDBRrrXcopeaZHY+XzdZaFyilYoF1Sql9nmjEX67oXZnA3F8VKaUGAjiXxSbH43ZKqUCMJP+q1vpt52q/P28ArfVxYAPGvRl/PefZwOVKqRyMsuuFSqlX8N/zPUVrXeBcFgPvYJSh3X7e/pLoe/Ik5GuAG52vbwRWmxiL2ynj0v1fQIbW+vEWb/nteSulYpxX8iilegEXAfvw03PWWv9Cax2ntU7A+L/7qdb6Ovz0fJsppXorpcKbXwOXAGl44Lz95slYpdQijDpf8yTkvzc5JLdTSr0OzMMYyrQI+A2wCngDGAIcAb6ttW59w9ZnKaXOAzYBqXxTv/0lRp3eL89bKTUB4yacHeNi7A2t9W+VUtH46Tk3c5Zufq61Xuzv56uUSsK4igejjP6a1vr3njhvv0n0Qggh2uYvpRshhBDtkEQvhBB+ThK9EEL4OUn0Qgjh5yTRCyGEn5NEL4QQfk4SvRBC+Ln/D+T71rFWvYG1AAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/memoryless_3_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "t_grid = np.linspace(0, 50, 100)\n", + "\n", + "class Erlang:\n", + "\n", + " def __init__(self, λ=0.5, n=10):\n", + " self.λ, self.n = λ, n\n", + "\n", + " def __call__(self, t):\n", + " n, λ = self.n, self.λ\n", + " return (λ**n * t**(n-1) * np.exp(-λ * t)) / factorial(n-1)\n", + "\n", + "e1 = Erlang(n=10, λ=0.5)\n", + "e2 = Erlang(n=10, λ=0.75)\n", + "\n", + "fig, ax = plt.subplots()\n", + "for e in e1, e2:\n", + " ax.plot(t_grid, e(t_grid), label=f'$n={e.n}, \\lambda={e.λ}$')\n", + "\n", + "ax.legend()\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The CDF of the Erlang distribution is\n", + "\n", + "$$\n", + " F(t) \n", + " = \\PP\\{W \\leq t\\}\n", + " = 1 - \\sum_{k=0}^{n-1} \\frac{(\\lambda t)^k}{k!} e^{-\\lambda t}\n", + "$$ (erlcdf)\n", + "\n", + "The Erlang distribution is of interest to us because of the following fact.\n", + "\n", + "```{proof:lemma} Distribution of Sum of Exponentials\n", + ":label: erlexp\n", + "\n", + "If, for some $\\lambda > 0$, the sequence $(W_i)$ is IID and exponentially\n", + "distributed with rate $\\lambda$, then $J_n := \\sum_{i=1}^n W_i$ has the Erlang\n", + "distribution with shape $n$ and rate $\\lambda$.\n", + "```\n", + "\n", + "This connects to Poisson process theory, as we shall soon see.\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "## Exercises\n", + "\n", + "### Exercise 1\n", + "\n", + "Due to its memoryless property, we can \"stop\" and \"restart\" an exponential\n", + "draw without changing its distribution.\n", + "\n", + "To illustrate this, we can think of fixing $\\lambda > 0$, drawing from\n", + "$\\Exp(\\lambda)$, and stopping and restarting whenever a threshold $s$ is crossed.\n", + "\n", + "In particular, consider the random variable $X$ defined as follows:\n", + "\n", + "* Draw $Y$ from $\\Exp(\\lambda)$.\n", + "* If $Y \\leq s$, set $X = Y$.\n", + "* If not, draw $Z$ independently from $\\Exp(\\lambda)$ and set $X = s + Z$.\n", + "\n", + "Show that $X \\sim \\Exp(\\lambda)$.\n", + "\n", + "\n", + "### Exercise 2\n", + "\n", + "Fix $\\lambda = 0.5$ and $s=1.0$.\n", + "\n", + "Simulate 1,000 draws of $X$ using the algorithm above.\n", + "\n", + "Plot the fraction of the sample exceeding $t$ for each $t \\geq 0$ (on a grid)\n", + "and compare to $t \\mapsto e^{-\\lambda t}$.\n", + "\n", + "Is the fit good? How about if the number of draws is increased?\n", + "\n", + "Are the results in line with those of the previous exercise?\n", + "\n", + "\n", + "## Solutions\n", + "\n", + "\n", + "### Solution to Exercise 1\n", + "\n", + "Let $X$ be constructed as in the statement of the exercise and fix $t > 0$.\n", + "\n", + "Notice that $X > s + t$ if and only if $Y > s$ and $Z > t$.\n", + "\n", + "As a result of this fact and independence,\n", + "\n", + "$$\n", + " \\PP\\{X > s + t\\}\n", + " = \\PP\\{Y > s \\} \\PP\\{Z > t\\}\n", + " = e^{-\\lambda(s + t)}\n", + "$$\n", + "\n", + "At the same time,\n", + "\n", + "$$\n", + " \\PP\\{X > s - t\\}\n", + " = \\PP\\{Y > s - t \\} \n", + " = e^{-\\lambda(s - t)}\n", + "$$\n", + "\n", + "Either way, we have $X \\sim \\Exp(\\lambda)$, as was to be shown.\n", + "\n", + "\n", + "### Solution to Exercise 2\n", + "\n", + "Here's one solution, starting with 1,000 draws." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deVxU9f7H8dd3hhn2RQVcwAVNRQRxQdy3XCt/LmWLmZUt2t6ve29Xu3VvdavbvbflV5m3xTSzRS1LszKXyiWXFNwFRHFHlE1BQNlmvr8/hriogKjAgeHzfDzmAXPOmTOfAXx75jvnfL5Ka40QQoj6z2R0AUIIIaqHBLoQQjgJCXQhhHASEuhCCOEkJNCFEMJJuBj1xP7+/rpNmzZGPb0QQtRL27Zty9BaB5S3zrBAb9OmDbGxsUY9vRBC1EtKqaMVrZMhFyGEcBIS6EII4SQk0IUQwkkYNoYuRH1SVFREcnIy+fn5RpciGgg3NzeCg4OxWCxVfowEuhBVkJycjLe3N23atEEpZXQ5wslprcnMzCQ5OZmQkJAqP+6yQy5KqblKqTSl1N4K1iul1DtKqSSl1G6lVPcrqFuIeiE/P58mTZpImItaoZSiSZMmV/yOsCpj6POAUZWsvwFoX3KbCrx3RRUIUU9ImIvadDV/b5cNdK31euB0JZuMBeZrh98AP6VU8yuupIqOxsew+b2HKDifW1NPIYQQ9VJ1nOUSBBwvcz+5ZNkllFJTlVKxSqnY9PT0q3qy7FOH6JO6gPiYX67q8UKImpWVlcV//vOf0vspKSlMmDCh0sccOXKE8PDwmi7tqtTl2i5WHYFe3vuCcmfN0Fp/qLWO0lpHBQSUe+XqZXWMHoFNKzL3SqALURddHOgtWrRg8eLFBlbUcFRHoCcDLcvcDwZSqmG/5XL1asQJt/b4pW3BZpfZlkTD8dlnnxEdHU3Xrl2ZNm0aNpuNmJgYunTpQn5+Pnl5eXTu3Jm9e/eydu1aBg4cyPjx4wkLC+Ohhx7CbrcDsGDBAiIiIggPD2f69Oml+/fy8uLZZ58lMjKS3r17k5qaCkB6ejq33HILPXv2pGfPnmzcuBGAF154gfvuu4/BgwfTtm1b3nnnHQBmzJjBwYMH6dq1K08//fQFR7hHjhxhwIABdO/ene7du7Np06bLvu7XXnuNnj170qVLF55//nkAlixZwrBhw9Bac/LkSTp06MCpU6fIzc1lypQpRERE0KVLF77++msAVq1aRZ8+fejevTu33norubmOIdtt27YxaNAgevTowciRIzl58mTp8sjISPr06cOsWbNKa6mo/rVr1zJ48GAmTJhAaGgokyZN4vfZ4GJiYujbty+RkZFER0eTk5ODzWbj6aefLn1dH3zwwdX8SVyiOk5bXAY8ppRaCPQCsrXWJ6thvxUqatWPiP2fse3gSaLbt6jJpxLiEi9+F0d8ytlq3WdYCx+e/5/OFa5PSEhg0aJFbNy4EYvFwiOPPMLnn3/O3XffzZgxY3juuec4f/48d911F+Hh4axdu5atW7cSHx9P69atGTVqFN988w19+/Zl+vTpbNu2jUaNGjFixAiWLl3KuHHjyMvLo3fv3rzyyiv8+c9/Zvbs2Tz33HM8+eSTPPXUU/Tv359jx44xcuRIEhISANi3bx9r1qwhJyeHjh078vDDD/PPf/6TvXv3snPnTsARgr8LDAxk9erVuLm5ceDAASZOnFhpT6dVq1Zx4MABtm7ditaaMWPGsH79esaPH8/XX3/NrFmzWLFiBS+++CLNmjVj+vTp+Pr6smfPHgDOnDlDRkYGL7/8Mj/99BOenp7861//4s033+SZZ57h8ccf59tvvyUgIIBFixbx7LPPMnfuXKZMmcLMmTMZNGgQTz/9dJXq37FjB3FxcbRo0YJ+/fqxceNGoqOjuf3221m0aBE9e/bk7NmzuLu7M2fOHHx9fYmJiaGgoIB+/foxYsSIKzpFsTyXDXSl1AJgMOCvlEoGngcsAFrr94HlwI1AEnAOmHJNFVVBi8hhuB74mISYX4huf1dNP50Qhvv555/Ztm0bPXv2BOD8+fMEBgYC8Le//Y2ePXvi5uZWepQMEB0dTdu2bQGYOHEiGzZswGKxMHjwYH4f8pw0aRLr169n3LhxWK1WRo8eDUCPHj1YvXo1AD/99BPx8fGl+z179iw5OTkA3HTTTbi6uuLq6kpgYGDpUX1FioqKeOyxx9i5cydms5n9+/dXuv2qVatYtWoV3bp1AyA3N5cDBw4wcOBAZs6cSXh4OL1792bixImltS5cuLD08Y0aNeL7778nPj6efv36AVBYWEifPn1ITExk7969DB8+HACbzUbz5s3Jzs4mKyuLQYMGATB58mR+/PHHy9YfHR1NcHAwAF27duXIkSP4+vrSvHnz0t+bj49P6evavXt36VBUdnY2Bw4cqPlA11pPvMx6DTx6TVVcIfd2/bGjKDr0K1pPktPJRK2q7Ei6pmitueeee3j11VcvWXf69Glyc3MpKioiPz8fT09P4NLT3pRSVDYpvMViKX2M2WymuLgYALvdzubNm3F3d7/kMa6urqXfl31MRf7v//6Ppk2bsmvXLux2O25ubpVur7XmmWeeYdq0aZesO3HiBCaTidTUVOx2OyaTCa31Ja9ba83w4cNZsGDBBcv37NlD586d2bx58wXLs7KyKsyUyuov72dRXj2/1zRz5kxGjhxZ6eu/UvWzl4u7H1m+nehcsJu4an7rK0RdNHToUBYvXkxaWhrgCPGjRx1dVKdOncpLL73EpEmTLhgT37p1K4cPH8Zut7No0SL69+9Pr169WLduHRkZGdhsNhYsWFB6JFqRESNG8O6775be/30opSLe3t6lR/AXy87Opnnz5phMJj799FNsNlul+xo5ciRz584tHfM+ceIEaWlpFBcXM2XKFL744gs6derEm2++WW6tZ86coXfv3mzcuJGkpCQAzp07x/79++nYsSPp6emlgV5UVERcXBx+fn74+vqyYcMGAD7//POrrj80NJSUlBRiYmIAyMnJobi4mJEjR/Lee+9RVFQEwP79+8nLy6t0X1VRPwMd8LhuIN1MB1i9u8LWwEI4jbCwMF5++WVGjBhBly5dGD58OCdPnmT+/Pm4uLhw5513MmPGDGJiYvjlF8cZYH369GHGjBmEh4cTEhLC+PHjad68Oa+++ipDhgwhMjKS7t27M3bs2Eqf+5133iE2NpYuXboQFhbG+++/X+n2TZo0oV+/foSHh18w/gzwyCOP8Mknn9C7d2/2799f+m6iIiNGjODOO++kT58+REREMGHCBHJycvjHP/7BgAEDGDBgAG+++SYfffQRCQkJPPfcc5w5c4bw8HAiIyNZs2YNAQEBzJs3j4kTJ9KlSxd69+7Nvn37sFqtLF68mOnTpxMZGUnXrl1LP+T8+OOPefTRR+nTp88F70yutH6r1cqiRYt4/PHHiYyMZPjw4eTn5/PAAw8QFhZG9+7dCQ8PZ9q0aZd9d1MVqrK3YDUpKipKX9MEF/uWw8KJ/MHjH7z551od8RENUEJCAp06dTK6jCpbu3Ytr7/+Ot9//73RpYhrUN7fnVJqm9Y6qrzt6+0ROq37oFEEZ2/nYLpcNSqEEPU30N0bURzQmV6mBFbGnTK6GiHqlMGDB8vReQNUfwMdsLQbSJT5AD/vOX75jYUQwsnV60CnTX9cKcTt5BZSss4bXY0QQhiqfgd6u+uxWb0Zb97IKhl2EUI0cPU70C3umMPHc6M5hl92HzG6GiGEMFT9DnSAyIl4cJ7GyatIPSvzPQpxpfr27XtV6yrzwgsv8Prrr19tSTWqLtd2rep/oLfsTZFPK8abNvDD7hrtCSaEUyqv4+HvV0BWpRuiqDvqf6CbTFgixtPPHMcvOytv9CNEfVZe+1xwtL2dPn06PXr0YNiwYWzdurW0pe2yZcsAmDdvHmPHjmXUqFF07NiRF198sXS/Xl5egONipCFDhnDnnXcSERFxwTqAf//730RERBAZGcmMGTMAmD17Nj179iQyMpJbbrmFc+fOVfoaKmrFO3bsWObPnw/ABx98wKRJk4Arbz1bXqtdgFdeeYWOHTsybNgwEhMTS5dXVP+9997LE088Qd++fWnbtu0F/dzL+zkcPHiQUaNG0aNHDwYMGMC+ffsu89usIVprQ249evTQ1ebYFq2f99GPP/OMPpaZV337FaJEfHz8f+8sn6713Bur97Z8+mWff/To0bqwsFBrrfXDDz+sP/nkE6211oBevny51lrrcePG6eHDh+vCwkK9c+dOHRkZqbXW+uOPP9bNmjXTGRkZ+ty5c7pz5846JiZGa621p6en1lrrNWvWaA8PD33o0KHS5/193fLly3WfPn10Xp7j31dmZqbWWuuMjIzSbZ999ln9zjvvaK21fv755/Vrr712yeuYOHGi/vXXX7XWWh89elSHhoZqrbU+deqUbteunV6/fr1u3769zszM1AUFBTokJERv3bpVa611dna2Lioq0h988IF+6aWXtNZa5+fn6x49euhDhw7plStX6gcffFDb7XZts9n0TTfdpNetW6djY2N1eHi4zsvL09nZ2bpdu3altVVU/z333KMnTJigbTabjouL0+3atav053D99dfr/fv3a621/u233/SQIUMq/X1W1QV/dyWAWF1BrlZHP3TjBUVh8whkhC2WH/ac5KFB7YyuSIhqVVn7XKvVyqhRjnncIyIicHV1xWKxEBERcUEv8uHDh9OkSRMAbr75ZjZs2EBU1IVXkEdHR5fbwvWnn35iypQpeHh4ANC4cWMA9u7dy3PPPUdWVha5ubmX7R5YUSvepk2b8ve//50hQ4awZMkSGjduzJ49e66o9WxFrXZzcnIYP358ae1jxowpff7K6h83bhwmk4mwsLDStsDl/Rxyc3PZtGkTt956a+ljCwoKKv051BTnCHSTCXOnmxi6fSETdx6RQBc164Z/1vpT6kra55Zte2symUrbuJpMpgsaPpXXTvdiFTWb0hW0gb333ntZunQpkZGRzJs3j7Vr11b6Oiprxbtnzx6aNGlCSkpKpc+pK2g9u3LlynJb7b711lsVtsOtrP6y7XB1Sc+r8mqy2+34+fldtgtlbaj/Y+i/6zQad32exqmbpLeLcDqVtc+tqtWrV3P69GnOnz/P0qVLSyd8qIoRI0Ywd+7c0jHm06dPA452sM2bN6eoqOiCNrOV7ae8Vrxbt27lxx9/ZMeOHbz++uscPnz4ilvPVtRqd+DAgSxZsoTz58+Tk5PDd999V/r8V1P/xT8HHx8fQkJC+OqrrwBH6O/ateuy+6oJzhPobQZit3oz0hzL97vkbBfhXCpqn3sl+vfvz+TJk+natSu33HLLJcMtlRk1ahRjxowhKiqKrl27lp7299JLL9GrVy+GDx9OaGjoZfdTXivegoICHnzwQebOnUuLFi144403uO+++7BYLFfUeraiVrvdu3fn9ttvL33dAwYMKK3nSuuv6Ofw+eefM2fOHCIjI+ncuTPffvttlX+21an+ts8tz+L7yY5fzQSPj1n1x+tlJiNRbepb+9yLzZs3j9jY2AuOjkXd13Da55Yn9CZ87dn4Ze5k36nyZ0wRQghn5VyB3n442mxllEss3+9OMboaIeqMe++9V47OGwDnCnRXb1TbwYy27uC7nSmVTogrxJWSvydRm67m7825Ah2gwyia2k5iyUpiV3K20dUIJ+Hm5kZmZqaEuqgVWmsyMzNxc3O7osc5x3noZXUYCT/ACJedLNnel64t/YyuSDiB4OBgkpOTSU9PN7oU0UC4ubkRHBx8RY9xvkD3DYamEdx8dg+37Urh2ZvCsLo43xsRUbssFku5V1AKUZc4Z9J1GEm7/Djs586wJjHN6GqEEKJWOGmgj8KkbYz12MPX25KNrkYIIWqFcwZ6UA/wa8U9XltZk5jG6bxCoysSQoga55yBbjJBlztomxNDY1sm3+2Sc9KFEM7POQMdIPIOlLYztdE2vt4uwy5CCOfnvIHepB0ERzPOvJHdydkcSJVWAEII5+a8gQ7QeRxNcvfT2pTBYjlKF0I4uSoFulJqlFIqUSmVpJSaUc56X6XUd0qpXUqpOKXUlOov9Sp0cMziMrX5AZbuOIHNLlf5CSGc12UDXSllBmYBNwBhwESlVNhFmz0KxGutI4HBwBtKKWs113rlmrSDxu0Y7rKT1LMFbEzKMLoiIYSoMVU5Qo8GkrTWh7TWhcBCYOxF22jAWzkakHsBp4Fi6oIOowjI2EozN5t8OCqEcGpVCfQg4HiZ+8kly8p6F+gEpAB7gCe11vaLd6SUmqqUilVKxdZaT4wOI1C2Ap5ok8yKvafIPldUO88rhBC1rCqBXt60PxcPRo8EdgItgK7Au0opn0sepPWHWusorXVUQEDAFRd7VVr3Aw9/bmI9BcV2luyQo3QhhHOqSqAnAy3L3A/GcSRe1hTgG+2QBBwGLj9BX20wWyDyDnyPrWZAC82CrcelBaoQwilVJdBjgPZKqZCSDzrvAJZdtM0xYCiAUqop0BE4VJ2FXpNuk8FezB8Cd5CYmsP2Y1lGVySEENXusoGutS4GHgNWAgnAl1rrOKXUQ0qph0o2ewnoq5TaA/wMTNda151TSgJDIbgnkRnf4Wk1sWDrMaMrEkKIalelfuha6+XA8ouWvV/m+xRgRPWWVs0i78D0wx+ZGlrAe7tT+OvoMHzdLUZXJYQQ1ca5rxQtq9MYUCYmem4jv8jO0h0njK5ICCGqVcMJdK9AaNOfwGM/EtHChwVbj8mHo0IIp9JwAh2g83jIPMDDnfLZdyqHHcflw1EhhPNoWIHeaQwoM8Ns6/G0mlmwRT4cFUI4j4YV6J7+0H4E1r1fMi6yKd/tTuFsvlw5KoRwDg0r0AG6T4bcUzzY7CD5RXaWbJcPR4UQzqHhBXr7EeAZSJvjS4gM9mX+5iPYpa2uEMIJNLxAL2kFwP4VTOvhxcH0PDZIW10hhBNoeIEOpa0ARhSvwd/LlXmbjhhdkRBCXLOGGegBHaBlL1x2fcGk6Jb8si+Nwxl5RlclhBDXpGEGOjiO0jP2c0+rNCxmxfzNR4yuSAghrknDDfTO48HiSePEhdwU0ZyvYpPJkVMYhRD1WMMNdFcvCB8Pe5dwf3QguQXFfL1NJr8QQtRfDTfQAbrdDUV5RGT9QrdWfnyy+aicwiiEqLcadqC3jAb/DrDjU+7t24bDGXmsO1BLc50KIUQ1a9iBrpTjw9HjW7gx8DSB3q7M23jE6KqEEOKqNOxAB+h2F7i4Ydn2EXf1bs26/ekcSM0xuiohhLhiEugejaHLbbBrEZMjfXCzmPhgfd2ZDlUIIapKAh0gehoUn6dR4iLu6NmKb3ee4GT2eaOrEkKIKyKBDtAsHFr3h5jZ3N+3FXYNc349bHRVQghxRSTQf9drGmQdo2XGekZ3ac6CrcfIPicXGgkh6g8J9N91vBF8gmHL+0wb2I68QhufbTlqdFVCCFFlEui/M7tA9ANweD1hpqMM6hDAxxsPk19kM7oyIYSoEgn0snrcC1Yv2Pg2Dw1qR0ZuIYulHYAQop6QQC/LvRFETYG9X9O70VkiW/ox+9dD2KQdgBCiHpBAv1jvR0CZUZtn8fCgthzNPMePe08aXZUQQlyWBPrFfFo4pqjb8SnDW5tp6+/JrDUHpWmXEKLOk0AvT78nobgA89YPeOz660g4eZbVCalGVyWEEJWSQC+Pf3voNBpiZjMm1JsQf0/e/ukAWstRuhCi7pJAr0ifxyA/G5fE73n8+uuIP3mWVfFylC6EqLuqFOhKqVFKqUSlVJJSakYF2wxWSu1USsUppdZVb5kGaNkLGreFXQsYE9mCEH9P3vrpgIylCyHqrMsGulLKDMwCbgDCgIlKqbCLtvED/gOM0Vp3Bm6tgVprl1IQORGO/IpLTjKPl4yly1G6EKKuqsoRejSQpLU+pLUuBBYCYy/a5k7gG631MQCtdVr1lmmQLrc5vu5aVHqU/vbPcpQuhKibqhLoQcDxMveTS5aV1QFopJRaq5TappS6u7oKNFSjNtBuKGx8C5fTSXKULoSo06oS6KqcZRcforoAPYCbgJHAX5VSHS7ZkVJTlVKxSqnY9PR6MnfnmJng4gpf3s2YMD85ShdC1FlVCfRkoGWZ+8FASjnbrNBa52mtM4D1QOTFO9Jaf6i1jtJaRwUEBFxtzbXLNwhung3pCbhsnll6lL4i7pTRlQkhxAWqEugxQHulVIhSygrcASy7aJtvgQFKKRellAfQC0io3lINdN1Q6HwzbHybsSGaDk29eG1lIkU2u9GVCSFEqcsGuta6GHgMWIkjpL/UWscppR5SSj1Usk0CsALYDWwFPtJa7625sg0w/EXQdsy/vMifR4ZyOCOPRTHHL/84IYSoJcqoqx+joqJ0bGysIc991X5+CX59HX3fKm5bbuNwxjnWPT0YT1cXoysTQjQQSqltWuuo8tbJlaJXov9T4NUUtfIvzBjVkYzcAuZukLlHhRB1gwT6lXD1gqHPw4lYemR+z8jOTflg/SEycwuMrkwIISTQr1jkRGgzAFY+yzN9vTlXWMzMX5KMrkoIISTQr5jJBGPfBW2nzcbp3B4VzOdbjnIs85zRlQkhGjgJ9KvRqA2M+DscWsMzTbdgNileX5VodFVCiAZOAv1q9bgPQgbis/5FnurpwbJdKWw7esboqoQQDZgE+tUymWDMu2Ar5D77Ypr6uPLid3HSEkAIYRgJ9GvRqDV0m4xl9wJeGOzH7uRsFm9PNroqIUQDJYF+rfo9CdrOqOyv6N7Kj3+vSCQnv8joqoQQDZAE+rVq1Bq63IbaPp+XRgaTmVcgpzEKIQwhgV4dej8MRefonPodt/YI5uONhzmUnmt0VUKIBkYCvTo0j4RWfWDrhzw9vD2uLmZe+j7e6KqEEA2MBHp16TUNso4ScGodTwy9jjWJ6fyyT2Y2EkLUHgn06hI6GjwDYOfn3Ns3hHYBnvx1aRznCouNrkwI0UBIoFcXswUiboPEFVgLs/jH+AhOZJ3n7Z8OGF2ZEKKBkECvTpF3gL0I4r6hV9sm3B7Vko82HCY+5azRlQkhGgAJ9OrULAICO8OOz0BrnrkxFD93C88s2YNNriAVQtQwCfTqpBT0mgopOyB2Dn4eVv46Ooxdx7P4fMtRo6sTQjg5CfTq1u1uaDcUVj4HGQcY27UFA9r78+8ViZzKzje6OiGEE5NAr24mE4z7j+ND0p//jlKKl8eFU2Sz88KyOKOrE0I4MQn0muDdDKKnQsJ3kJ5I6yaePDmsPSviTvHD7pNGVyeEcFIS6DWl9yNgcYdf3wRg6oC2RAb78tzSPaTnyBykQojqJ4FeUzybQNR9sOdLOLEdF7OJN26LJK/QxrNL9qC1nPUihKheEug1adCfwTMQlj0BtiKuC/TmTyM6sCo+lSU7ThhdnRDCyUig1yQ3X7jpDUjdA1veB+D+/m2Jat2I55fFyVkvQohqJYFe0zqNhrZDYOPbUHTeMaH0rZEU2zTTv94tQy9CiGojgV4bBj4NeemwfT4Abfw9mXFDKOv2p7Mw5rjBxQkhnIUEem1o08/RL33j23DuNACTe7em33VN+Pt38SSl5RhcoBDCGUig15bhf4e8DPh0PORnYzIp3rytK+5WM499sYP8IpvRFQoh6jkJ9NrSMhpu/xRS42DOSMhIoqmPG2/cGsm+Uzm8ujzB6AqFEPWcBHpt6jASJn0FeWkwewgkxzIkNJD7+4fwyeajrIo7ZXSFQoh6rEqBrpQapZRKVEolKaVmVLJdT6WUTSk1ofpKdDLthsDUdeDRBD67GU7u5s+jOtK5hQ9//no3J7PPG12hEKKeumygK6XMwCzgBiAMmKiUCqtgu38BK6u7SKfj1xLuWQZWb1h0F67Fecyc2I3CYjtPLtwpvdOFEFelKkfo0UCS1vqQ1roQWAiMLWe7x4GvgbRqrM95+bWCCXMg+ziseIa2AV68PC6crYdP8/qqRKOrE0LUQ1UJ9CCg7MnSySXLSimlgoDxwPuV7UgpNVUpFauUik1PT7/SWp1Pq97Q/w+w8zNIXMHN3YOZGN2K99YeZMVeGU8XQlyZqgS6KmfZxWMCbwHTtdaVnnuntf5Qax2ltY4KCAioao3ObdB0CAyDH/4A+Wd5YUwYkS39+NNXu0hKyzW6OiFEPVKVQE8GWpa5HwykXLRNFLBQKXUEmAD8Ryk1rloqdHYuVhgzE86mwLLHcNWFvDepO64uJqZ9GktuQbHRFQoh6omqBHoM0F4pFaKUsgJ3AMvKbqC1DtFat9FatwEWA49orZdWe7XOKjgKhr8I8d/CnOG0sJ9i5p3dOJyRx9Nf7ZJ+L0KIKrlsoGuti4HHcJy9kgB8qbWOU0o9pJR6qKYLbDD6PQl3fglZx+DDQfS172DGDaH8uPcUH6w/ZHR1Qoh6QBl19BcVFaVjY2MNee467fRh+HIypMaj/+dtHtvXmeV7TjJ7chTDwpoaXZ0QwmBKqW1a66jy1smVonVN4xCY8iOEDEQte4w3Ox+hS5AvTyzcQVxKttHVCSHqMAn0usjV2zH8EtQD1x+fYs74IPzcLdw/L5bUszIphhCifBLodZWLFW6eDbZC/Fc+zNyJoeTkF/HAJ7GcK5QzX4QQl5JAr8uatIOxsyA5htAVd/D+uBbEpWTz1KKd2KU9gBDiIhLodV34zTBxIWQmMeDnW5jZ9xwr41J5+YcEOZ1RCHEBCfT6oP1weOBncPPhxm0PsqjlN3yxcR//WXvQ6MqEEHWIBHp90TQMpq5FRT9Ir/TF/Oj3GrNXxrJw6zGjKxNC1BES6PWJqzfc+Brc9iltig7yrfe/eW7JLmnkJYQAJNDrp7AxqHH/oXXRQR4O2MMTC3ew+WCm0VUJIQwmgV5fdb4ZAkL5X+sy2jRy48H5sWw/dsboqoQQBpJAr69MJuj/B8wZCXw54BRNvKzcM2cru45nGV2ZEMIgEuj1Wfgt0Lwrfj/9ia/GeePnaWHynC3sSZYWAUI0RBLo9ZnZxXGOupsvgd/dzcLJHfFxt3DXnC3S90WIBkgCvb7zaQ53fA65aQT9+hcWPNALL1cX7vpoC/EpZ42uTjDgQ7YAABHsSURBVAhRiyTQnUGLbjDkLxC/lJZ7ZrLg/ijcLGbu+HCzfFAqRAMige4s+j3pOPNl7au0Wjqer6Z0oZGnlbs+2sLGpAyjqxNC1AIJdGdhMsOEuXDzR3BiG8ExL/PVtD60bOTBlI9jWBUnFx8J4ewk0J2JUtDlVuj3BGybR+C+T1n0YE86tfDh4c+3s2RHstEVCiFqkAS6MxryLLTuB8v/hN/861kwxpvoNo15atEuZq8/JF0ahXBSEujOyMUV7vkebp0H507j8ckIPul5hBsjmvHK8gSeXxaHTfqpC+F0JNCdlckEncfDtPUQ1B3rt9OY5beAh/q3ZP7mo0z7VGY+EsLZSKA7O++mcPe30OcxVMxsZpz6I6+N9OeXfWnc8eFvpOXIHKVCOAsJ9IbAbIGRrziGYNLiuXXnA3xycwsOpOYy7t2N7D0hV5UK4Qwk0BuSzuPh3u8hP5sBm+/jm8lt0cCE9zfx7c4TRlcnhLhGEugNTYtucNdiyEml0+rJfHdfKBFBvjy5cCev/pggH5YKUY9JoDdELaPhzkVw5gj+C2/ki2FF3NW7FR+sO8SUeTFknSs0ukIhxFWQQG+oQgY4PixFYflsDC/n/5N3h3uy+WAGN72zgR3SA0aIekcCvSFr1Rse3gSDZsChdYzefDs/D01BKbj1/c189KtchCREfSKB3tBZPWDIM/DEdgjuSav1f2St7wvMCN7Lyz8k8OD8bTIEI0Q9IYEuHLwCYfJSuPF1XOzFPJD2Cks7rWHd/lRufPtXmYRaiHpAAl38l9kFoh90XF3abTJdD89mW8hsgs2nufOj33jlh3jyi2xGVymEqECVAl0pNUoplaiUSlJKzShn/SSl1O6S2yalVGT1lypqjdkFxsyEUf/EJ3ULi4r/l7fabuOjXw8ybtZGEk7KTEhC1EWXDXSllBmYBdwAhAETlVJhF212GBikte4CvAR8WN2FilqmFPR+GB7ZjAqOYuyJN4gNmU1hTiZj3t3ArDVJFNnsRlcphCijKkfo0UCS1vqQ1roQWAiMLbuB1nqT1vr389x+A4Krt0xhmEZtSsfWm6RuYrX3i4zv6M5rKxMZ8+5G9iRL2wAh6oqqBHoQcLzM/eSSZRW5H/ixvBVKqalKqVilVGx6enrVqxTGUsoxtn7PMsw5J/i3aRYf3NWNzNwCxs7awKvLEzhfKGPrQhitKoGuyllW7snJSqkhOAJ9ennrtdYfaq2jtNZRAQEBVa9S1A2t+8KoVyFpNSP3/Imf727O7T1b8sH6Q4x6ez1rEtOMrlCIBq0qgZ4MtCxzPxhIuXgjpVQX4CNgrNZaznFzVlH3w9C/weF1eM8bzKsRaXzxYC/MJsWUj2N4cH4sx0+fM7pKIRqkqgR6DNBeKRWilLICdwDLym6glGoFfANM1lrvr/4yRZ2hFAz4IzyxEwI6wqJJ9E1bxIq7WjB9VCgbkzIY9uY63v7pgJziKEQtU1W5tFspdSPwFmAG5mqtX1FKPQSgtX5fKfURcAtwtOQhxVrrqMr2GRUVpWNjY6+peGGwvEz44lY4sc1xP6gHp7s/yl8TQ/hh90mC/Nx5emRHxkS2wGQqb+ROCHGllFLbKsrXKgV6TZBAdyKnD0Picoj9GDIPQMcbSQy8kWf3BhJ7spjwIB/+ckMn+l7nb3SlQtR7EuiidtiKYfNMWP8GFOagPQPZGjaDP+xuw4nsfAZ3DGDGDaGENvMxulIh6i0JdFG7bEVw7DdY+Rc4tRt74+tY13QyTyZ0IregmPHdgnn8+uto4+9pdKVC1DuVBbr0chHVz2xx9Ft/cA2Mex+Tuy9DEp5na5/fuL9fG77fncLQN9fxhy93cjgjz+hqhXAacoQuap6tGL5/EnZ8Bm0Hc7rPX/gw3sK82FQKi+2M7RrEo0Ou47pAL6MrFaLOkyEXYTytIXYOrH4eCnMBRWHrQSyz3sDf9rXifLFmdJcWTB3QlohgX6OrFaLOkkAXdcfZk3B0I6TGwe4v4WwyxYERrHUbyttHW7OnoCm9Qhrz4IC2XB8aKKc7CnERCXRRN9ltjlDf8H+QkQhAqk8Ec88N5LPc7jT19+e+/iHc0j0Yd6vZ4GKFqBsk0EXdl3Uc4r+F7fMhIxGNIsPkz3eFPfjMcgtDo8KZGN2KtgEyzi4aNgl0UX9oDce3wsFf0GnxsO8HlLaRpv34ydadTUFTuKFfT4aHNcXqIidpiYZHAl3UXxlJEL+E/JR4LInLsGn4rHgYP7sOJbpbN0b36kQ7OWoXDYgEunAOWcewr/0XatcXKO2YLSlLe5Jv9sbN3QPX8P/BPfoeaNLO4EKFqDkS6MK5nDkKJ3eSc+ogxw4mcCotDUt+Jv1MezErTWZAL7yHPY21wzBHd0ghnIgEunB68SlnWf3bdlz2LmKcbSVBKpMjbmHkhN9F+yGTcfOU/jHCOUigiwaj2GZnY+IJzvw6h8iTiwghhVztTpzvQNw6DqX9gFvx8GlsdJlCXDUJdNEgFRXbiPttFbbYebTL2oAfuZzXVuI8euLWrAP+Ax6gWdtwo8sU4opIoIsGr6i4mPiYNRRt+4ymmVtpak+lACtvez6JJWIcQzs1pVurRpjlylRRx0mgC1GG1ppjh/fjvmQKgTlxbLWHstwWTaqlJdaQPkR1aEnf6/xp6++Jkg9VRR0jgS5EeYoLYfsn2Ne/jin3lGMRJnbZ27HR3plE9254tutDr/ZB9LvOn2a+bgYXLIQEuhCV0xryMiAtDn1oPQUH1mJN24lJ28jHQqytA5vsnTnsHYVXSBTdQwKIat2IdgFe0jxM1DoJdCGuVP5ZOLoJfWgtBQfW4nY6AYBzuHHY3pTjOpBT5mbkBETh2mEwke1aEhHsi4fVxeDChbOTQBfiWuWmw5Ff0cc2cz41iaLMI3jkJWPRhaRrXx4rfIIYOnFdoBcRQX50CfYlItiXsOY+uFmkU6SoPhLoQtSE4kI4thnb939AnTlMsk839ulWnMopZntBMJvsnck0NaZ9oBcRQb6ENvchtJk3HZt54+/lanT1op6SQBeiJuWfhV/fgKSf4cxhtK0IZSsAINO9DTvMEWzIa0VWgeY03hzXgeR7BNGueWM6lgR8p2Y+tG/qJUfz4rIk0IWoTXY7pO6BQ+vg8Ho4ugmKLpwM244i09SE/cXNWWrrwzpbJBnKlxZ+nrQN8KKtvyftAkq+D/CkmY+bnEIpAAl0IYxVXAhnkx1Bn5cGZ444GoxlHUUnx6AykwAoNLmT5BbOQVsgZ/OLKbJpcnHnhPZHma14+fhibtIWl2adCWriQ6vGHrRs7EFzXzdczNIbvqGQQBeirtIakmPg5C5IT4QjGyD3FBrQdo0qzEFp2wUPydOuHNBB2DGRphuRTAA5bi3w8PTB28sTa+PWeDW7joAWLWnm50mgtysWCXynUVmgyzlWQhhJKWgZ7biVXVxyw1YMualgL4b8bMhMwv3IBjqmHaSgsIh2uafwOLcbS1EBZOG4JQO7IV9bOKH9iccdzBayrM3Icw+iyLslqnEbXAPa4tM0hOaNvGjq4ybztjoBCXQh6jKzC/gG/fd+8y6Ywm/GHXD/fZnWkJcOxQVQdI7CjMNknUwiP+0Q1qxj+BTkYSvMp1nBfppkbcAlywbHHQ8t1iZO6ibs0AGcNDXltLUFue4tKPQKxtPTEy8vH6xNWtPI1xd/b1f8vVzx97Li5eoiY/p1kAS6EPWdUuAVWHrXGtCRwE4VbGsrhrMnOJ9+iJyTBylIP4Q+c4S2OceJOLcb76I1UAScrfjp0rUPCQSx2xzONs8BnPHugJ+7lUaeFnzdrTTysNDIw4pvyddGHhZ8PSz4uVtlHtgaJmPoQoj/KjwHWccgOxnsRdjOZ3M+/Qjnzp/nXKGN8wVFkJuK79lEmuXtw4SdYy6tOUIQh22BHC32pdBecWgXmT3JsLTghHt73Nw98XZzcdxcLXiVfO/l6oKPm8XxvZsL3iXfe7u64G4142F1adBdMWUMXQhRNVYPCAx13AAz4FVyu8S507D7S1odWEWrM0cYmB0L5kLHgypjg8I8K0cLr+PU2QDy7C7k2KwcLW7MaZsLp6tQpotJYXUxYTWbsbgorGYTRRZvct2a42Kx4uZiws1ixtVixs3FhKvFcV95NKHYNwQ3qwuuLiZcLWZcXUy4WUy4uphxLfnqZjFhNZvq3bBSlY7QlVKjgLdx/Ko+0lr/86L1qmT9jcA54F6t9fbK9ilH6EI4Gbsdzp92jOmXS8P5LMg84Dib59QeyD4Odpvj4qyC7Fop85RuxH57MOn4UvLRc4UKlSt7TJ1IMzfDWvIfh8XFhMVswtXFhItJYXKxkObeDmXxwGI2YS1Z//t/NGUf4/hPSNGpuQ9dgv2uqv5rOkJXSpmBWcBwHJ+fxyillmmt48tsdgPQvuTWC3iv5KsQoqEwmcDTv/JtvAIhoAOE3nTpuvxsxxj/VdFw/oxjqOii0zyLbJrCYjv5xTZsZ47hnvwbPbIOYc4/gtaO/vi6ZBca7ViGY7m1OJeJtp/AhuNWgUJcSFFNsaNK9vnffZQnOWQCTPn7Vb7WilVlyCUaSNJaHwJQSi0ExgJlA30sMF87qv9NKeWnlGqutT5Z7RULIZyTm++1Pd7TH/zbX7LYUnLzLF3yUNX3abdD6l44l1HxNkXnsR7bTJusY5escoS6I9jtGuxaY9eaoA6hVa/hClQl0IMoPckJcBylX3z0Xd42QcAFga6UmgpMBWjVqtWV1iqEELXLZILmXS6/XXnvOChzPQGX/2ihOlTlHKLyBpkufh9RlW3QWn+otY7SWkcFBARUpT4hhBBVVJVATwZalrkfDKRcxTZCCCFqUFUCPQZor5QKUUpZgTuAZRdtswy4Wzn0BrJl/FwIIWrXZcfQtdbFSqnHgJU4hoHmaq3jlFIPlax/H1iO45TFJBynLU6puZKFEEKUp0oXFmmtl+MI7bLL3i/zvQYerd7ShBBCXAlprCCEEE5CAl0IIZyEBLoQQjgJw7otKqXSgaNX+XB/oJJLt5ySvOaGQV5zw3Atr7m11rrcC3kMC/RroZSKrag5jbOS19wwyGtuGGrqNcuQixBCOAkJdCGEcBL1NdA/NLoAA8hrbhjkNTcMNfKa6+UYuhBCiEvV1yN0IYQQF5FAF0IIJ1HvAl0pNUoplaiUSlJKzTC6npqmlGqplFqjlEpQSsUppZ40uqbaoJQyK6V2KKW+N7qW2lIy09dipdS+kt93H6NrqklKqadK/qb3KqUWKKXcjK6pJiil5iql0pRSe8ssa6yUWq2UOlDytVF1PFe9CvQy85veAIQBE5VSYcZWVeOKgT9qrTsBvYFHG8BrBngSSDC6iFr2NrBCax0KROLEr18pFQQ8AURprcNxdHK9w9iqasw8YNRFy2YAP2ut2wM/l9y/ZvUq0Ckzv6nWuhD4fX5Tp6W1Pqm13l7yfQ6Of+RBxlZVs5RSwcBNwEdG11JblFI+wEBgDoDWulBrnWVsVTXOBXBXSrkAHjjppDha6/XA6YsWjwU+Kfn+E2BcdTxXfQv0iuYubRCUUm2AbsAWYyupcW8BfwbsRhdSi9oC6cDHJUNNHymlPC/3oPpKa30CeB04hmPu4Wyt9Spjq6pVTX+fBKjka2B17LS+BXqV5i51RkopL+Br4H+11meNrqemKKVGA2la621G11LLXIDuwHta625AHtX0NrwuKhkzHguEAC0AT6XUXcZWVf/Vt0BvkHOXKqUsOML8c631N0bXU8P6AWOUUkdwDKldr5T6zNiSakUykKy1/v3d12IcAe+shgGHtdbpWusi4Bugr8E11aZUpVRzgJKvadWx0/oW6FWZ39SpKKUUjnHVBK31m0bXU9O01s9orYO11m1w/H5/0Vo7/ZGb1voUcFwp1bFk0VAg3sCSatoxoLdSyqPkb3woTvwhcDmWAfeUfH8P8G117LRKU9DVFRXNb2pwWTWtHzAZ2KOU2lmy7C8l0wIK5/I48HnJwcohnHhuXq31FqXUYmA7jjO5duCkLQCUUguAwYC/UioZeB74J/ClUup+HP+53VotzyWX/gshhHOob0MuQgghKiCBLoQQTkICXQghnIQEuhBCOAkJdCGEcBIS6EII4SQk0IUQwkn8P/cHpVXVTuBVAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/memoryless_5_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "λ = 0.5 \n", + "np.random.seed(1234)\n", + "t_grid = np.linspace(0, 10, 200)\n", + "\n", + "@njit\n", + "def draw_X(s=1.0, n=1_000):\n", + " draws = np.empty(n)\n", + " for i in range(n):\n", + " Y = np.random.exponential(scale=1/λ)\n", + " if Y <= s:\n", + " X = Y\n", + " else:\n", + " Z = np.random.exponential(scale=1/λ)\n", + " X = s + Z\n", + " draws[i] = X\n", + " return draws\n", + "\n", + "fig, ax = plt.subplots()\n", + "draws = draw_X()\n", + "empirical_exceedance = [np.mean(draws > t) for t in t_grid]\n", + "ax.plot(t_grid, np.exp(- λ * t_grid), label='exponential exceedance')\n", + "ax.plot(t_grid, empirical_exceedance, label='empirical exceedance')\n", + "ax.legend()\n", + "\n", + "plt.show()\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The fit is already very close, which matches with the theory in Exercise 1.\n", + "\n", + "The two lines become indistinguishable as $n$ is increased further." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhU9dn/8fedmUxWkkAIEAi7CIRAEBJWEawscXkAK1YRraJVqUv7+KsKVh+t1VZrrbVa6664AhZ3iwouiIIKQXaQHSSsCUsISSbLzP37IzEFzAYkOZnJ/bquuZI5c2bOZ0L45OSbc75HVBVjjDGBL8TpAMYYY+qGFboxxgQJK3RjjAkSVujGGBMkrNCNMSZIuJ3acMuWLbVTp05Obd4YYwLS0qVLc1Q1obLHHCv0Tp06kZmZ6dTmjTEmIInI9qoesyEXY4wJElboxhgTJKzQjTEmSDg2hm5MICkpKSErKwuv1+t0FNNEhIeHk5SURGhoaK2fY4VuTC1kZWXRrFkzOnXqhIg4HccEOVVl//79ZGVl0blz51o/r8YhFxF5QUT2icjqKh4XEXlMRDaJyEoR6XcCuY0JCF6vl/j4eCtz0yBEhPj4+BP+jbA2Y+jTgYxqHj8X6FZ+uw548oQSGBMgrMxNQzqZ77caC11VFwAHqlllHPCylvkGiBORxBNOUkvb1y3lmyevp8hbUF+bMMaYgFQXR7m0A3YcdT+rfNlPiMh1IpIpIpnZ2dkntbFDuzczaO9Mvv/6g5N6vjGmfh06dIh//etfFfd37drFhAkTqn3Otm3bSElJqe9oJ6UxZzteXRR6Zb8XVHrVDFV9RlXTVDUtIaHSM1dr1GPIBRzRCIpWvntSzzfG1K/jC71t27bMnj3bwURNR10UehbQ/qj7ScCuOnjdSoWFR/J9zGC6HVyAr7S0vjZjTKPz6quvMmDAAPr27cv111+Pz+djyZIl9OnTB6/XS35+Pr169WL16tXMnz+fs846iwsvvJDk5GSmTJmC3+8HYMaMGfTu3ZuUlBSmTp1a8frR0dHceeedpKamMmjQIPbu3QtAdnY2F110Eenp6aSnp7Nw4UIA/vCHP3D11VczYsQIunTpwmOPPQbAtGnT2Lx5M3379uW22247Zg9327ZtDBs2jH79+tGvXz8WLVpU4/v+61//Snp6On369OGee+4B4O2332bkyJGoKrt37+b0009nz549HDlyhMmTJ9O7d2/69OnDm2++CcDcuXMZPHgw/fr14+KLL+bIkSMALF26lOHDh9O/f3/GjBnD7t27K5anpqYyePBgnnjiiYosVeWfP38+I0aMYMKECfTo0YNJkybx49XglixZwpAhQ0hNTWXAgAHk5eXh8/m47bbbKt7X008/fTLfEj9RF4ctvgfcJCIzgYFArqruroPXrVrP/6H54s9Yt2QuPQefV6+bMuZ4976/hrW7Dtfpaya3jeGe/+lV5ePr1q1j1qxZLFy4kNDQUG644QZee+01fvnLXzJ27FjuuusuCgsLufzyy0lJSWH+/PksXryYtWvX0rFjRzIyMnjrrbcYMmQIU6dOZenSpTRv3pzRo0fzzjvvMH78ePLz8xk0aBB/+tOfuP3223n22We56667+O1vf8stt9zCmWeeyQ8//MCYMWNYt24dAN9//z2ff/45eXl5dO/enV//+tc8+OCDrF69muXLlwNlJfijVq1aMW/ePMLDw9m4cSMTJ06sdk6nuXPnsnHjRhYvXoyqMnbsWBYsWMCFF17Im2++yRNPPMFHH33EvffeS5s2bZg6dSqxsbGsWrUKgIMHD5KTk8P999/PJ598QlRUFH/5y1945JFHuOOOO7j55pt59913SUhIYNasWdx555288MILTJ48mccff5zhw4dz22231Sr/smXLWLNmDW3btmXo0KEsXLiQAQMGcMkllzBr1izS09M5fPgwERERPP/888TGxrJkyRKKiooYOnQoo0ePPqFDFCtTY6GLyAxgBNBSRLKAe4BQAFV9CpgDnAdsAgqAyaeUqBZ6DPs5Rd/eTu6yt8EK3TQBn376KUuXLiU9PR2AwsJCWrVqBcDdd99Neno64eHhFXvJAAMGDKBLly4ATJw4ka+++orQ0FBGjBjBj0OekyZNYsGCBYwfPx6Px8MFF1wAQP/+/Zk3bx4An3zyCWvXrq143cOHD5OXlwfA+eefT1hYGGFhYbRq1apir74qJSUl3HTTTSxfvhyXy8WGDRuqXX/u3LnMnTuXM844A4AjR46wceNGzjrrLB5//HFSUlIYNGgQEydOrMg6c+bMiuc3b96cDz74gLVr1zJ06FAAiouLGTx4MOvXr2f16tWMGjUKAJ/PR2JiIrm5uRw6dIjhw4cDcMUVV/Dhhx/WmH/AgAEkJSUB0LdvX7Zt20ZsbCyJiYkV/24xMTEV72vlypUVQ1G5ubls3Lix/gtdVSfW8LgCN55SihMU3SyOZZFpdNr3Ger3IyE2g4FpONXtSdcXVeXKK6/kgQce+MljBw4c4MiRI5SUlOD1eomKigJ+etibiFDdReFDQ0MrnuNyuSgtH9L0+/18/fXXRERE/OQ5YWFhFZ8f/Zyq/P3vf6d169asWLECv99PeHh4teurKnfccQfXX3/9Tx7buXMnISEh7N27F7/fT0hICKr6k/etqowaNYoZM2Ycs3zVqlX06tWLr7/++pjlhw4dqvKQweryV/a1qCzPj5kef/xxxowZU+37P1EB24TF3c6jDTlsXvmV01GMqXfnnHMOs2fPZt++fUBZiW/fXjaL6nXXXcd9993HpEmTjhkTX7x4MVu3bsXv9zNr1izOPPNMBg4cyBdffEFOTg4+n48ZM2ZU7IlWZfTo0fzzn/+suP/jUEpVmjVrVrEHf7zc3FwSExMJCQnhlVdewefzVftaY8aM4YUXXqgY8965cyf79u2jtLSUyZMn8/rrr9OzZ08eeeSRSrMePHiQQYMGsXDhQjZt2gRAQUEBGzZsoHv37mRnZ1cUeklJCWvWrCEuLo7Y2Fi++qqsW1577bWTzt+jRw927drFkiVLAMjLy6O0tJQxY8bw5JNPUlJSAsCGDRvIz8+v9rVqI2ALvduwiynVEHKWvOl0FGPqXXJyMvfffz+jR4+mT58+jBo1it27d/Pyyy/jdru57LLLmDZtGkuWLOGzzz4DYPDgwUybNo2UlBQ6d+7MhRdeSGJiIg888ABnn302qamp9OvXj3HjxlW77ccee4zMzEz69OlDcnIyTz31VLXrx8fHM3ToUFJSUo4Zfwa44YYbeOmllxg0aBAbNmyo+G2iKqNHj+ayyy5j8ODB9O7dmwkTJpCXl8ef//xnhg0bxrBhw3jkkUd47rnnWLduHXfddRcHDx4kJSWF1NRUPv/8cxISEpg+fToTJ06kT58+DBo0iO+//x6Px8Ps2bOZOnUqqamp9O3bt+KPnC+++CI33ngjgwcPPuY3kxPN7/F4mDVrFjfffDOpqamMGjUKr9fLr371K5KTk+nXrx8pKSlcf/31Nf52UxtS3a9g9SktLU1P9QIXq/48nNjSHDrcvaaOUhlTuXXr1tGzZ0+nY9Ta/Pnzefjhh/ngAztfI5BV9n0nIktVNa2y9QN2Dx3gSOcMOvizyNqwzOkoxhjjuIAu9E7DLsWvwu5FM2te2ZgmZMSIEbZ33gQFdKEnJnVmbWgvWu340OkoxhjjuIAudID9nc6jo287OZur/8u7McYEu4Av9A5nlg277Fw0o+aVjTEmiAV8oXfu1JUV7l7Eb58DDh2xY4wxjUHAFzrA/o7nkVT6Awe2rXA6ijEBZ8iQISf1WHX+8Ic/8PDDD59spHrVmLOdqqAo9M7DJuJTYceXr9W8sjHmGJXNePjjGZC1mQ3RNB5BUehdO3dhpbs3LW3YxQSxyqbPhbJpb6dOnUr//v0ZOXIkixcvrpjS9r333gNg+vTpjBs3joyMDLp37869995b8brR0dFA2clIZ599Npdddhm9e/c+5jGAhx56iN69e5Oamsq0adMAePbZZ0lPTyc1NZWLLrqIgoLqryRW1VS848aN4+WXXwbg6aefZtKkScCJTz1b2VS7AH/605/o3r07I0eOZP369RXLq8p/1VVX8Zvf/IYhQ4bQpUuXY+Zzr+zrsHnzZjIyMujfvz/Dhg3j+++/r+Ffs56oqiO3/v37a1369JUHVO+J0b0bMuv0dY1RVV27du1/78yZqvrCeXV7mzO1xu1fcMEFWlxcrKqqv/71r/Wll15SVVVA58yZo6qq48eP11GjRmlxcbEuX75cU1NTVVX1xRdf1DZt2mhOTo4WFBRor169dMmSJaqqGhUVpaqqn3/+uUZGRuqWLVsqtvvjY3PmzNHBgwdrfn6+qqru379fVVVzcnIq1r3zzjv1scceU1XVe+65R//617/+5H1MnDhRv/zyS1VV3b59u/bo0UNVVffs2aNdu3bVBQsWaLdu3XT//v1aVFSknTt31sWLF6uqam5urpaUlOjTTz+t9913n6qqer1e7d+/v27ZskU//vhjvfbaa9Xv96vP59Pzzz9fv/jiC83MzNSUlBTNz8/X3Nxc7dq1a0W2qvJfeeWVOmHCBPX5fLpmzRrt2rVrtV+Hn/3sZ7phwwZVVf3mm2/07LPPrvbfs7aO+b4rB2RqFb1aF/OhNwpdh1+Gb+OD7Fz4Oq269Xc6jjF1qrrpcz0eDxkZZddx7927N2FhYYSGhtK7d+9j5iIfNWoU8fHxAPz85z/nq6++Ii3t2DPIBwwYUOkUrp988gmTJ08mMjISgBYtWgCwevVq7rrrLg4dOsSRI0dqnD2wqql4W7duzR//+EfOPvts3n77bVq0aMGqVatOaOrZqqbazcvL48ILL6zIPnbs2IrtV5d//PjxhISEkJycXDEtcGVfhyNHjrBo0SIuvvjiiucWFRVV+3WoL0FT6B3bd2CZJ5VWP8wBfRjsCu2mvpz7YINvUquZPvfoaW9DQkIqpnENCQk5ZsKnyqbTPV5Vk01pFdPAXnXVVbzzzjukpqYyffp05s+fX+37qG4q3lWrVhEfH8+uXbuq3aZWMfXsxx9/XOlUu48++miV0+FWl//o6XC1fCi3skx+v5+4uLgaZ6FsCEExhv6jw10uoJ1/F7vWL3Y6ijF1qrrpc2tr3rx5HDhwgMLCQt55552KCz7UxujRo3nhhRcqxpgPHDgAlE0Hm5iYSElJyTHTzFb3OpVNxbt48WI+/PBDli1bxsMPP8zWrVtPeOrZqqbaPeuss3j77bcpLCwkLy+P999/v2L7J5P/+K9DTEwMnTt35t///jdQVvorVjhzxF1QFXq34ZdSqiHsXmgnGZngUtX0uSfizDPP5IorrqBv375cdNFFPxluqU5GRgZjx44lLS2Nvn37Vhz2d9999zFw4EBGjRpFjx49anydyqbiLSoq4tprr+WFF16gbdu2/O1vf+Pqq68mNDT0hKaerWqq3X79+nHJJZdUvO9hw4ZV5DnR/FV9HV577TWef/55UlNT6dWrF+++68xF7AN6+tzKLPvzz2hXsp1Wd60HV9CMKBmHBdr0ucebPn06mZmZx+wdm8avSU2fW5kDPSfRSnPYvdgufGGMaVqCrtB7n30pOzQB39dPOh3FmEbjqquusr3zJiDoCr1VXBRfxI4n6fAydLdNBWDqjlPDk6ZpOpnvt6ArdICwAVdSpG72L3zZ6SgmSISHh7N//34rddMgVJX9+/cTHh5+Qs8Lyr8annNGd76Y25fB698B/0MQ4nI6kglwSUlJZGVlkZ2d7XQU00SEh4eTlJR0Qs8JykJvEeVha5vzGL3vj/i2fInrtBFORzIBLjQ0tNIzKI1pTIJyyAWg89CLOKLh7F30qtNRjDGmQQRtoY/o1YHPZSBx2+ZAqTPzKhhjTEMK2kL3uEM42HUckf588td85HQcY4ypd0Fb6ACpZ40jR2PY//UrTkcxxph6F9SF3qdDPF95htFmz3zwHnY6jjHG1KugLnQRwZcyAQ8l5GTaVADGmOAW1IUOMGR4Bjs0gSOZM52OYowx9apWhS4iGSKyXkQ2ici0Sh6PFZH3RWSFiKwRkcl1H/XkJMZF8l3MSNofWoz/8F6n4xhjTL2psdBFxAU8AZwLJAMTRST5uNVuBNaqaiowAvibiHjqOOtJi0q7FBd+fviy5gnsjTEmUNVmD30AsElVt6hqMTATGHfcOgo0k7JrM0UDB4BSGokhg89kvXZAVs+ueWVjjAlQtSn0dsCOo+5nlS872j+BnsAuYBXwW1X1H/9CInKdiGSKSGZDzokR6XGzqfW5dCxcQ8GeTQ22XWOMaUi1KfTKrq56/JRzY4DlQFugL/BPEYn5yZNUn1HVNFVNS0hIOOGwp6LD8MsB2PTZ9AbdrjHGNJTaFHoW0P6o+0mU7YkfbTLwlpbZBGwFar5AXwNKSe7NKlcyzTe/AzYFqjEmCNWm0JcA3USkc/kfOi8F3jtunR+AcwBEpDXQHdhSl0FPlYiQ12087X072Lz6W6fjGGNMnaux0FW1FLgJ+BhYB7yhqmtEZIqITClf7T5giIisAj4FpqpqTn2FPlm9Rv6SEnWxZ8GLTkcxxpg6V6v50FV1DjDnuGVPHfX5LmB03Uare7EtE1kRM5Se+/5DYWEhERERTkcyxpg6E/Rnih4vfMBkWkgey+a97nQUY4ypU02u0E8fMpZ9Ek/EajvJyBgTXJpcoYvLTVani0gt+o5NG9c6HccYY+pMkyt0gK6jrgdgx2fPOZzEGGPqTpMs9Ni2p7Ehqj/dd79LobfY6TjGGFMnmmShA4SkXUlbcsj8/C2noxhjTJ1osoXebdgvyJVmhCx7GbUzR40xQaDJFrqEhrOzw3jSi75h1QabsMsYE/iabKEDdBo1BY/42PzJ805HMcaYU9akCz0yKYUdUSn03vcue3MLnY5jjDGnpEkXOkDE4Gs4TXbx5cd28QtjTGBr8oXectAkDrla0HbtcxSV+pyOY4wxJ63JFzruMA6lTGYIy1nw5RdOpzHGmJNmhQ50HHMThYQh3zxhhzAaYwKWFTogkS3Y3uEizvJ+zqrvv3c6jjHGnBQr9HIdz/8dLvGzZ+5jTkcxxpiTYoVeLqL1aWxoPoKBB95l9959TscxxpgTZoV+lBZjbidW8vn+3b86HcUYY06YFfpRWvcYwqroM+m/81VyD2Q7HccYY06IFfpxojLuJkYK2PDOA05HMcaYE2KFfpwuKQPJjBjK6T/Mwpuf63QcY4ypNSv0SoQN/3/EcoRV7//T6SjGGFNrVuiVSBl4DmvcvWi//kV8JXZFI2NMYLBCr4SI4E2/kTaazcq5052OY4wxtWKFXoW+Iy9luyQR892TqN/vdBxjjKmRFXoVXC4Xu3tdS1ffFtYufN/pOMYYUyMr9Gr0Pf86coiDrx5xOooxxtTICr0a4RGRbDztGnoVLef7b+Y4HccYY6plhV6D1AtvKdtL//zPYFPrGmMaMSv0GkRGNWNDt+voUbSK9d986HQcY4ypkhV6LfS98LfkEEfRFw87HcUYY6pUq0IXkQwRWS8im0RkWhXrjBCR5SKyRkSC6lpukZHRbO5yBX28S1m7dIHTcYwxplI1FrqIuIAngHOBZGCiiCQft04c8C9grKr2Ai6uh6yO6j3+/3GESA5/YnvpxpjGqTZ76AOATaq6RVWLgZnAuOPWuQx4S1V/AFDVoLtCRGRMCzZ2vIT0ggWsXrnU6TjGGPMTtSn0dsCOo+5nlS872ulAcxGZLyJLReSXlb2QiFwnIpkikpmdHXjzjXcfdzul4ib7Y7sAhjGm8alNoUsly44/fs8N9AfOB8YA/ycip//kSarPqGqaqqYlJCSccFinRbZoy6Z24xlyZB4r1q5zOo4xxhyjNoWeBbQ/6n4SsKuSdT5S1XxVzQEWAKl1E7Fx6TJ2Gm7xs+0/D6N2XLoxphGpTaEvAbqJSGcR8QCXAu8dt867wDARcYtIJDAQCMpd2IjWp7EjcTQ/O/Ifvlixyek4xhhTocZCV9VS4CbgY8pK+g1VXSMiU0RkSvk664CPgJXAYuA5VV1df7Gd1e6C39NMCtn84T/w+W0v3RjTOIhTwwZpaWmamZnpyLbrQvaTF+Das4wvMuZy4eBeTscxxjQRIrJUVdMqe8zOFD1JLcfdT5zkU/DJX/CW+JyOY4wxVugnS9r2JafrRUwo/Q9vf/ql03GMMcYK/VS0Gnc/hLiJ/+YBcgtKnI5jjGnirNBPRUwiuf1uZDTf8N4Hs51OY4xp4qzQT1GrMbdyyJ1A3zUPsWP/EafjGGOaMCv0U+WJhHPuprds4dOZjzmdxhjThFmh14G4gZezp1kK5+17mm/WbXM6jjGmibJCrwshITS/6BFaySG2vXM/pT6/04mMMU2QFXodCes0kJ0dxnKh9x3en7/I6TjGmCbICr0Otb3oQTQkhOgv/8ihgmKn4xhjmhgr9Dokse3IS7uZUXzLW2/NdDqOMaaJsUKvYwmjb+VgaBsGbXiY73cddDqOMaYJsUKva6ERhJ57P8kh21k44y/4bTZGY0wDsUKvB9FnTGB3wlAuO/w8c+YvcDqOMaaJsEKvDyK0ueI5SkPC6LTgFrJz851OZIxpAqzQ64nEtCV/1EOksJmFr/7R6TjGmCbACr0etRk8kU0thjNm3wssXhq4F/MwxgQGK/T6JELSpCfwiZuQ/9yCt7jU6UTGmCBmhV7PwuPbs3fgHaT5V/LZzEedjmOMCWJW6A2g65ib2BLZhyGbH2H9+nVOxzHGBCkr9IYQEkLLy54hVHyUvjGZoiKv04mMMUHICr2BxCT1ZOugP9PLt47lL93mdBxjTBCyQm9AKRnX8HXzsQzc9TJbFr3ldBxjTJCxQm9gva5+gg3SiZbzfoP3wE6n4xhjgogVegOLaRZD7vnP4PF7+eHl60FtrhdjTN2wQndAetpAPmt7Lacf+pItn093Oo4xJkhYoTvkrF/+gdUhpxO/4C7ycrKcjmOMCQJW6A6JjghDxv2LcC1iy/TrUb9dh9QYc2qs0B3UKzWdpV1+TeqRr1j6tp1Faow5NVboDhs46R5WhPWnz8o/sWP1QqfjGGMCWK0KXUQyRGS9iGwSkWnVrJcuIj4RmVB3EYOby+2mzeRX2C9xRL55OUXZW5yOZIwJUDUWuoi4gCeAc4FkYKKIJFex3l+Aj+s6ZLBr3aYd2zNewuUvIv+5C+DIPqcjGWMCUG320AcAm1R1i6oWAzOBcZWsdzPwJmBtdBIGDTqT2d0fIdybw+HnxoI31+lIxpgAU5tCbwfsOOp+VvmyCiLSDrgQeKruojU9V/ziYh6K/T0RBzdQ+PIvoKTQ6UjGmABSm0KXSpYdf3rjo8BUVfVV+0Ii14lIpohkZmdn1zZjkxHmdnHt1ddzd8jNhO36ltI3rgKfXRTDGFM7tSn0LKD9UfeTgF3HrZMGzBSRbcAE4F8iMv74F1LVZ1Q1TVXTEhISTjJycGsXF8EFk27m3pIrcW/8CF34D6cjGWMCRG0KfQnQTUQ6i4gHuBR47+gVVLWzqnZS1U7AbOAGVX2nztM2EUNPa0mbUb9hjm8AvvkPwv7NTkcyxgSAGgtdVUuBmyg7emUd8IaqrhGRKSIypb4DNlVThnfhiy63UuBzc3jmtTaeboypkahDs/2lpaVpZmamI9sOFHneEh599EH+z/swBZ3HEHn56+ByOx3LGOMgEVmqqmmVPWZnijZizcJDmXTNLTzA1URu/Zjit28Am/PFGFMFK/RGrktCNMOvuJNHSyfgWT0L/0d32BzqxphKWaEHgCFdW9J27D28UJpByOKn0PkPOB3JGNMIWaEHiF8M6MDeIXfz79KzkC/+Aov+6XQkY0wjY4UeQKZmJPPZ6XcxxzcA5t4JS6c7HckY04hYoQeQkBDhb5f25/lWv2eBPxV9/39h+etOxzLGNBJW6AEm0uPmmclD+HPMnXxLCvrODbBiltOxjDGNgBV6AIqPDuO5a4YxNfT3LJVe6DtTYOW/nY5ljHGYFXqASmoeybO/GsZNOpXlIcno29fB6jedjmWMcZAVegA7vXUznph8JlcX38oaV0/0zWthzdtOxzLGOMQKPcD179iCRyYNZVLhraxz9UBnXwNr33U6ljHGAVboQeDsHq146LIhTCz4Hevd3dF/T4aVbzgdyxjTwKzQg8SYXm14cOIQLsn/HWtCe6FvXQeLn3U6ljGmAdnUfUHk3N6J+HQIE2a4eCXmKdLn3AqFh2D4bU5HM8Y0ACv0IHNBn7b4/AO4bJaL5+KiGf75/eCJgsE3OB3NGFPPrNCD0Li+7RARrp0VwvToAgZ//Hskth0kj3M6mjGmHtkYepAam9qWJ69I5/qCKax1dS87+mXDx07HMsbUIyv0IHZOz9Y8PXko15Tczno6oDMvh3UfOB3LGFNPrNCD3JCuLXnq2nO41n8na7Qj+sYVsOxVp2MZY+qBFXoT0Ld9HM9OGcVN7nv42p8C794IC//hdCxjTB2zQm8ierSJ4fUbR/JA3B/4j28QzLsb5v6fXc7OmCBihd6EtI2L4LVfn8XMDvfwSulIWPQY+v5vrdSNCRJW6E1MTHgoz08exPLe/8cTpWOR716i9EO78LQxwcAKvQnyuEN4+BeplAy/ixdLx+Be/CT5H1ipGxPorNCbKBHhf0d1J/GSv/OajiFq6ZPkzLzBSt2YAGaF3sRl9G5H/ynP8pr757Rc/zprXp/mdCRjzEmyQjf0SIzl/FueYn5kBr02PsU3j1+FNz/X6VjGmBNkhW4AiIsK48xbXmFJm0sZkPMOuX9LY/eyD52OZYw5AVbopoI71EP6lKdZPnIGhX4Xie9eyvbpvwKv7a0bEwis0M1P9Bt2Lp6bFvF25ASSts7m4N8G4M1a5XQsY0wNrNBNpdq2bMEFv3uWV1OepbjYS+lzo9n89XtOxzLGVKNWhS4iGSKyXkQ2ichPDoMQkUkisrL8tkhEUus+qmlooa4Qrrz4Ynb8/H12SwIdP7qSz159gNJSn9PRjDGVqLHQRcQFPAGcCyQDE0Uk+bjVtgLDVbUPcB/wTF0HNc5JS+1Dq9/MZ0N0Oj/b9CBLHzqP7Vs3OB3LGHOc2kBbzqoAAA6/SURBVOyhDwA2qeoWVS0GZgLHXPpGVRep6sHyu98ASXUb0zgttnkLkn/3IatTppJavJRW04eS+eKtlBYVOB3NGFOuNoXeDthx1P2s8mVVuQao9Hg3EblORDJFJDM7O7v2KU3jEOIiZcLvybtmIaubDSFt+7PsfiidLd996nQyYwy1K3SpZFml54eLyNmUFfrUyh5X1WdUNU1V0xISEmqf0jQqCe27k37ruyw58zncPi+d3r2IJU9dT+GRw05HM6ZJq02hZwHtj7qfBOw6fiUR6QM8B4xT1f11E880ZukjLybit0v4On486XtmcuBvaXw373XU73c6mjFNUm0KfQnQTUQ6i4gHuBQ45vg1EekAvAVcoar217ImJK55C4b+Zjprx8xExUW/hb9m+4Pp7NywzOloxjQ5NRa6qpYCNwEfA+uAN1R1jYhMEZEp5avdDcQD/xKR5SKSWW+JTaOUPPhcWk9bzoLke4kuyib2tQw+fPkvHD6S53Q0Y5oMUYemS01LS9PMTOv9YJSzcwtHXp1Ep8K1HCSGjadfR9+LbsMTFu50NGMCnogsVdW0yh6zM0VNnWvZrgudbl/ElvNmkOXpyoAND3PgwRTWvP0QWnTE6XjGBC0rdFM/ROgy4DxSpn3GiuHPkxPSkl4r/kTegz3Z+fbdkG9/Nzemrlmhm3olISGknj2BHr9fxKeDX2YZ3Wm34h8UPdyTnW/eAUU2xm5MXbExdNOgCot9/OfTz4he/CgZ+hV50ozc3leRNOZ/Iaql0/GMafSqG0O3QjeOKCz2MXfeHOIy/8FwXUIxHvaf/gvaZNyKtOjsdDxjGi0rdNNoeUt8zPlsPu5v/kmG/wtcouxuN4bW595OaFI/p+MZ0+hYoZtGz1vi46Ovl+H96gnOK/qQGCkkq/kAmo+8lajk0SCVzUBhTNNjhW4Cht+vfLl6M1mfPMk5uW/SRg6yJ6IbMvQ3tB48EVyhTkc0xlFW6CYgrflhHys/fI70Xa9ymuzkQEg82d0n0vn8W/FEN3c6njGOsEI3AS37cCHfzp1BwtqXGehfxmGiWNPuF3QcOYW2nXs4Hc+YBmWFboKCz6989+183F8+RGr+14SIsirsDLw9LiJ52DiiWnZwOqIx9c4K3QSdPTs2sXXeM3Te8RZtNBu/CitjhqODb6LPwHNwueycOROcrNBN0FK/n3Urvmbf1zNI2zebaArZRQLr2v6c+LOupc/ppxESYkfImOBhhW6aBG/eAdbPn0Homtkke7/Dp8LKkJ7kJI6g9aBfkJLS18rdBDwrdNPk5O9YSdaXrxG5bR7tizfjV+FT12Cy22fQoe/PSEtJJjzU5XRMY06YFbpp0o7s3cquT/5J+80ziPDnA5Cp3dnW/Eyiep5D3wHDSWwe7XBKY2rHCt0YgNJiinetYNeSD4jY+D6tvZsBOKyRfOdJ52CXC0jsfwFndGlNmNv23k3jZIVuTCU0by97Vszj0Jq5tNvzOTF6mMMawVeaSlb8UFzJY0nv2YlebWNx2di7aSSs0I2pia+E/PWfcWjJGzTLmk9MSQ5FGsoifzLfuXrjTRpKux4D6N8pgZ6JzXDbYZHGIVboxpwIVdi1jIKlr+Pb8CnNjmwBIFcj+dbfk0zpTW6bQbTu2pf+nVtyRoc4YsJtjhnTMKzQjTkVeXtg65fkr/8Mtn1JVP4OAHI0hm/8PVni78Gh5r1p1rEvPTu0ok+7OLq3aYbHbXvxpu5ZoRtTlw79AFu/pGTzfPybFxBWuAeAElys97dnpb8LaziN/JYpxHTsQ0pSS1LaxdK1VZT9sdWcMit0Y+qLKhzeCTu/Q3d+h/eHpbj2LMdTchgAr4ayRjux2t+JLSRRENsVT5uetG3XkZ5tY+jeJoa2seGIzfduaqm6Qnc3dBhjgooIxCZBbBKSPJYIKCv5A1tg1zLCspaS/EMmqdkLcZcWQD6wGQ5timK9tmeuvyNb3F3wxicT2S6Fjq2a0yUhiq4J0bSLi7AzW80JsT10YxqCKhzeBTnrIXsDxXvWUrxzFeEH1uL2eYGyIZuN/iS2aCJ7tTk50oLCmM4UtupLQpsOdEmIoktCNB1aRNI8MtT26pso20M3xmkiENuu7Nb1Z3gAD4DfBwe2wp4VuHev4rSdy+l8YBvu/BWE+grL9ui3Qu6WKLZrK37Q1nyhiex0taegWWdo2Y1W8fG0bxFB++aRtG8RSfsWEUR67L92U2R76MY0RqpQdBj2rYOd3+HL2YR332bkwBbC87MIwV+x6i6NZ7M/kd0aTw6xZGssXk88rphWRDSLx9O8HTHxibSJiyAxNoLE2HBaxYTZH2gDlO2hGxNoRCA8FjoMgg6DcAFRPz5WWlQ2Rp+zAXI2kJizkYR9G9DD63EX5hCipaBAbvktCwo0jB2awA5NYI22IksTyPe0whXdgrBm8YTHtiQypiWxsc1p2Sycls3CaBkdRstoD9FhbhveCRBW6MYEGncYtOpZdgMEqDitye8H7yE4sg/ys8s+P7wLd/Zm2uVsI+nQNobnfUmor6Cs9PPKb7vKnl6gYezVOPbSghXanL3anP0ST2F4K0ojE/BENy//AZBATHQz4iI9NI8MJS4ylLhID3ERocRGhNqZtA6xQjcmmISEQGSLshv/vd5qxZg9lA3nFByAI3vLCr/wIBQewpefAwd30eLgTlrk7SElfwfh3kzc/mIo4b97/IBfhZ3akr0055BGsY1osjWOXdqCPdqCYk8c7vAowsKjCI+MJiyyGRHRzYmKiqRZmJtm4W6ahYcSHV72eUy4m+iwUJqFu4n0uOw3gpNkhW5MUyMCUfFlt6O4gMjj11UtK/y83WV7/d5DUHgIydtDm+wNJBzOxl94gJDCzXgK95UN9/zIW3479N9FBRrGYSIpVA9ePBRR9jFLPRSWf+7Fg98Vjt8VgYaGQ2gEIaGRhHgiyj73RODyROEKi8AdFkVoWCShEVF4wqMIi4giPCKayHAPER4XUR43ER4XYe6QJvFDolaFLiIZwD8o+zd/TlUfPO5xKX/8PKAAuEpVv6vjrMaYhiby3z3+1r3+u5iyYZ5jZrDx+6EgB3Kzyv6gW1IIJQVlH4vzwXuYiMKDuAsOUerNx1dciL+4AC0pREu8SOkBpLQIl68Ql7+IUJ8Xt6+07IfCCSpWF17C8OLhcPkPj2IJoyQkjBIJozgkjNKQMEpdEfhCwvC5wlF3eNkPEncEEhqOhkYi7jBcoR5cbg9udyiu0DDcoR7coR5CQz24PR5C3aG4KpaFVXwM9XjKbm43HldIg5xTUGOhi4gLeAIYBWQBS0TkPVVde9Rq5wLdym8DgSfLPxpjmoqQEIhuVXargnDc8E9NfKVQWggl3vKPZTctKaDEW0BRYT7F3nxKvfmUFBVQWlSAv7gAX3EhWv7DgpJCpLSQ8FIvUT4vLp8Xt/8Qbn8RnlIvoVpMmHoJpbTmPCehVEMoJLTit5Fi8fBD50s468p763xbtdlDHwBsUtUtACIyExgHHF3o44CXtewYyG9EJE5EElV1d50nNsY0HS43uJpBWLNjFp/wD4ba8Pug1FvxQ+PHHxylJSUUlxRTUlxMcXERpSXFlJYUUVJSgq+kGL+vGF9pKVpajN9Xgr+0BPUdfSsmpNSL+IoIKf+BEpfQri6TV6hNobcDdhx1P4uf7n1Xtk474JhCF5HrgOsAOnTocKJZjTGm/oS4wBNVditX6dBSI1abY4sqG/g5/myk2qyDqj6jqmmqmpaQkFCbfMYYY2qpNoWeBbQ/6n4SFUetntA6xhhj6lFtCn0J0E1EOouIB7gUeO+4dd4DfillBgG5Nn5ujDENq8YxdFUtFZGbgI8pO2zxBVVdIyJTyh9/CphD2SGLmyg7bHFy/UU2xhhTmVodh66qcygr7aOXPXXU5wrcWLfRjDHGnAibcMEYY4KEFboxxgQJK3RjjAkSjl3gQkSyge0n+fSWQE4dxgkE9p6bBnvPTcOpvOeOqlrpiTyOFfqpEJHMqq7YEazsPTcN9p6bhvp6zzbkYowxQcIK3RhjgkSgFvozTgdwgL3npsHec9NQL+85IMfQjTHG/FSg7qEbY4w5jhW6McYEiYArdBHJEJH1IrJJRKY5nae+iUh7EflcRNaJyBoR+a3TmRqCiLhEZJmIfOB0loZSfqWv2SLyffm/92CnM9UnEbml/Ht6tYjMEJFwpzPVBxF5QUT2icjqo5a1EJF5IrKx/GPzuthWQBX6Udc3PRdIBiaKSLKzqepdKfA7Ve0JDAJubALvGeC3wDqnQzSwfwAfqWoPIJUgfv8i0g74DZCmqimUzeR6qbOp6s10IOO4ZdOAT1W1G/Bp+f1TFlCFzlHXN1XVYuDH65sGLVXdrarflX+eR9l/8vq5IGEjISJJwPnAc05naSgiEgOcBTwPoKrFqnrI2VT1zg1EiIgbiCRIL4qjqguAA8ctHge8VP75S8D4uthWoBV6VdcubRJEpBNwBvCts0nq3aPA7YDf6SANqAuQDbxYPtT0nIhE1fSkQKWqO4GHgR8ou/ZwrqrOdTZVg2r940WAyj+2qosXDbRCr9W1S4ORiEQDbwL/q6qHnc5TX0TkAmCfqi51OksDcwP9gCdV9Qwgnzr6NbwxKh8zHgd0BtoCUSJyubOpAl+gFXqTvHapiIRSVuavqepbTuepZ0OBsSKyjbIhtZ+JyKvORmoQWUCWqv7429dsygo+WI0EtqpqtqqWAG8BQxzO1JD2ikgiQPnHfXXxooFW6LW5vmlQERGhbFx1nao+4nSe+qaqd6hqkqp2ouzf9zNVDfo9N1XdA+wQke7li84B1joYqb79AAwSkcjy7/FzCOI/AlfiPeDK8s+vBN6tixet1SXoGouqrm/qcKz6NhS4AlglIsvLl/2+/LKAJrjcDLxWvrOyhSC+Nq+qfisis4HvKDuSaxlBOgWAiMwARgAtRSQLuAd4EHhDRK6h7IfbxXWyLTv13xhjgkOgDbkYY4ypghW6McYECSt0Y4wJElboxhgTJKzQjTEmSFihG2NMkLBCN8aYIPH/Ae0D/+0EJXv6AAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/memoryless_7_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "fig, ax = plt.subplots()\n", + "draws = draw_X(n=10_000)\n", + "empirical_exceedance = [np.mean(draws > t) for t in t_grid]\n", + "ax.plot(t_grid, np.exp(- λ * t_grid), label='exponential exceedance')\n", + "ax.plot(t_grid, empirical_exceedance, label='empirical exceedance')\n", + "ax.legend()\n", + "\n", + "plt.show()\n" + ] + } + ], + "metadata": { + "jupytext": { + "formats": "ipynb,md:myst", + "text_representation": { + "extension": ".md", + "format_name": "myst", + "format_version": "0.9", + "jupytext_version": "1.5.0" + } + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.7" + }, + "source_map": [ + 13, + 43, + 49, + 340, + 364, + 455, + 483, + 489 + ] + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file diff --git a/_sources/memoryless.md b/_sources/memoryless.md new file mode 100644 index 0000000..b481a07 --- /dev/null +++ b/_sources/memoryless.md @@ -0,0 +1,502 @@ +--- +jupytext: + formats: ipynb,md:myst + text_representation: + extension: .md + format_name: myst + format_version: '0.9' + jupytext_version: 1.5.0 +kernelspec: + display_name: Python 3 + language: python + name: python3 +--- + + +# Memoryless Distributions + + +## Overview + + +Markov processes are, by definition, forgetful. + +In particular, for any Markov processes, the distribution over future outcomes +depends only on the current state, rather than the entire history. + +In the case of continuous time Markov chains, which jump between discrete +states, this requires that the amount of elapsed time since the last jump is +not helpful in predicting the timing of the next jump. + +In other words, the jump times are "memoryless". + +It is remarkable that the only distribution on $\RR_+$ with this +property is the exponential distribution. + +Similarly, the only memoryless distribution on $\ZZ_+$ is the geometric +distribution. + +This lecture tries to clarify these ideas. + +We will use the following imports: + +```{code-cell} ipython3 +import numpy as np +import matplotlib.pyplot as plt +import quantecon as qe +from numba import njit +from scipy.special import factorial, binom +``` + + +## The Geometric Distribution + +Consider betting on a roulette wheel and suppose that red has come up four times in a row. + +Since five reds in a row is an unlikely event, many people instinctively feel +that black is more likely on the fifth spin --- "Surely black will come up this time!" + +But rational thought tells us such instincts are wrong: the four previous reds make no +difference to the outcome of the next spin. + +(Many casinos offer an unlimited supply of free alcoholic beverages in order to discourage this kind of rational analysis.) + +A mathematical restatement of this phenomenon is: the geometric distribution is memoryless. + + +### Memorylessness + +Let $X$ be a random variable supported on the nonnegative integers $\ZZ_+$. + +We say that $X$ is [geometrically distributed](https://en.wikipedia.org/wiki/Geometric_distribution) if, for some $\theta$ satisfying $0 \leq \theta \leq 1$, + +$$ + \PP\{X = k\} = (1-\theta)^k \theta + \qquad (k = 0, 1, \ldots) +$$ (geodist) + +An example can be constructed from the discussion of the roulette wheel above. + +Suppose that, + +* the outcome of each spin is either red or black, +* spins are labeled by $0, 1, 2, \ldots$, +* on each spin, black occurs with probability $\theta$ and +* outcomes across spins are independent. + +Then {eq}`geodist` is the probability that the first occurrence of black is at spin $k$. + +(The outcome "black" fails $k$ times and then succeeds.) + +Consistent with our discussion in the introduction, the geometric distribution +is memoryless. + +For example, given any nonnegative integer $m$, we have + +$$ + \PP \{X = m + 1 \,|\, X > m \} = \theta +$$ (memgeo) + +In other words, regardless of how long we have seen only red outcomes, the +probability of black on the next spin is the same as the unconditional +probability of getting black on the very first spin. + +To show this, we note that the left hand side is + +$$ + \frac{ \PP \{X = m + 1 \text{ and } X > m \} } + {\PP \{X \geq m\}} + = + \frac{ \PP \{X = m + 1 \} } + {\PP \{X > m\}} + = \frac{ (1-\theta)^{m+1} \theta } + {\sum_{k > m} (1-\theta)^k \theta } +$$ + +The right hand side simplifies to $\theta$, completing the proof of {eq}`memgeo`. + + + +## The Exponential Distribution + + +Later, when we construct continuous time Markov chains, we will need to +specify the distribution of the holding times, which are the time intervals +between jumps. + +As discussed above (and again below), the holding time distribution must be +memoryless, so that the chain satisfies the Markov property. + +While the geometric distribution is memoryless, its discrete support makes it +a poor fit for the continuous time case. + +Hence we turn to the [exponential distribution](https://en.wikipedia.org/wiki/Exponential_distribution), which is supported on $\RR_+$. + +A random variable $Y$ on $\RR_+$ is called **exponential with rate $\lambda$**, denoted by $Y \sim \Exp(\lambda)$, if + +$$ + \PP\{Y > y\} = e^{-\lambda y} + \qquad (y \geq 0) +$$ + + + +(geomtoexp)= +### From Geometric to Exponential + +The exponential distribution can be regarded as the "limit" of the geometric +distribution. + +To illustrate, let us suppose that + +* customers enter a shop at discrete times $t_0, t_1, \ldots$ +* these times are evenly spaced, so that $h = t_{i+1} - t_i$ for some $h > 0$ and all $i \in \ZZ_+$ +* at each $t_i$, either zero or one customers enter (no more because $h$ is small) +* entry at each $t_i$ occurs with probability $\lambda h$ and is independent over $i$. + +The fact that the entry probability is proportional to $h$ is important in +what follows. + +You can imagine many customers passing by the shop, each entering +independently. + +If we halve the time interval, then we also halve the probability that a +customer enters. + +Let + +* $Y$ be the time of the first arrival at the shop, +* $t$ be a given positive number and +* $i(h)$ be the largest integer such that $t_{i(h)} \leq t$. + +Note that, as $h \to 0$, the grid becomes finer and $t_{i(h)} = i(h) h \to t$. + +Writing $i(h)$ as $i$ and using the geometric distribution, the probability that +the first arrival occurs after $t_{i}$ is $(1-\lambda h)^{i}$. + +Hence + +$$ + \PP\{Y > t_{i} \} + = (1-\lambda h)^i + = \left( 1- \frac{\lambda i h}{i} \right)^i +$$ + +Using the fact that $e^x = \lim_{i \to \infty}(1 + x/i)^i$ for all $x$ and $i +h = t_i \to t$, we obtain, for large $i$, + +$$ + \PP\{Y > t\} + \approx + e^{- \lambda t} +$$ + +In this sense, the exponential is the limit of the geometric distribution. + + +### Memoryless Property of the Exponential Distribution + +The exponential distribution is the only memoryless distribution supported on $\RR_+$, as the next theorem attests. + +```{proof:theorem} Characterization of the Exponential Distribution +:label: exp_unique + +If $X$ is a random variable supported on $\RR_+$, then there exists a +$\lambda > 0$ such that $X \sim \Exp(\lambda)$ if and only if, for all +positive $s, t$, + +$$ + \PP \{X > s + t \,|\, X > s \} = \PP \{X > t\} +$$ (memexpo) + +``` + +```{proof:proof} + +To see that {eq}`memexpo` holds when $X$ is exponential with rate $\lambda$, +fix $s, t > 0$ and observe that + +$$ + \frac{ \PP \{X > s + t \text{ and } X > s \} } + {\PP \{X > s\}} + = + \frac{ \PP \{X > s + t \} } + {\PP \{X > s\}} + = \frac{e^{-\lambda s - \lambda t}}{e^{-\lambda s}} + = e^{-\lambda t} +$$ + +To see that the converse holds, let $X$ be a random variable supported on $\RR_+$ +such that {eq}`memexpo` holds. + +The "exceedance" function $f(s) := \PP\{X > s\}$ then has three properties: + +1. $f$ is decreasing on $\RR_+$, +1. $0 < f(t) < 1$ for all $t > 0$, +1. $f(s + t) = f(s) f(t)$ for all $s, t > 0$. + +The first property is common to all exceedance functions, the second is due to +the fact that $X$ is supported on all of $\RR_+$, and the +third is {eq}`memexpo`. + +From these three properties we will show that + +$$ + f(t) = f(1)^t \;\; \forall \, t \geq 0 +$$ (implex) + +This is sufficient to prove the claim because then $\lambda := - \ln f(1)$ is a positive real number (by property 2) and, moreover, + +$$ + f(t) + = \exp( \ln ( f(1) ) t) + = \exp( - \lambda t) +$$ + +To see that {eq}`implex` holds, fix positive integers $m,n$. + +We can use property 3 to obtain both + +$$ + f(m/n) = f(1/n)^m + \quad \text{and} \quad + f(1) = f(1/n)^n +$$ + +It follows that $f(m/n)^n = f(1/n)^{m n} = f(1)^m$ and, raising to the power +of $1/n$, we get {eq}`implex` when $t=m/n$. + +The discussion so far confirms that {eq}`implex` holds when $t$ is rational. + +So now take any $t \geq 0$ and rational sequences $(a_n)$ and $(b_n)$ +converging to $t$ with $a_n \leq t \leq b_n$ for all $n$. + +By property 1 we have $f(a_n) \leq f(t) \leq f(b_n)$ for all $n$, so + +$$ + f(1)^{a_n} \leq f(t) \leq f(1)^{b_n} + \quad \forall \, n \in \NN +$$ + +Taking the limit in $n$ completes the proof. +``` + +(fail_mem)= +### Failure of Memorylessness + +We know from the proceeding section that any distribution on $\RR_+$ other +than the exponential distribution fails to be memoryless. + +Here's an example that helps to clarify (although the support of the distribution is a proper subset of $\RR_+$). + +A random variable $Y$ has the Pareto distribution with positive parameters $t_0, \alpha$ if + +$$ + f(t) + := \PP\{Y > t\} + = + \begin{cases} + 1 & \text{ if } t \leq t_0 + \\ + (t_0 / t)^\alpha & \text{ if } t > t_0 + \end{cases} +$$ + +As a result, with $s > t_0$, + +$$ + \PP \{Y > s + t \,|\, Y > s \} + = + \frac{ \PP \{Y > s + t \} } + {\PP \{Y > s\}} + = \left( \frac{t}{t + s} \right)^\alpha +$$ + +Since this probability falls with $s$, the distribution is not memoryless. + +If we have waited many hours for an event (i.e., $s$ is large), then the +probability of waiting another hour is relatively small. + + +## Sums of Exponentials + +A random variable $W$ on $\RR_+$ is said to have the [Erlang +distribution](https://en.wikipedia.org/wiki/Erlang_distribution) if its +density has the form + +$$ + f(t) = \frac{\lambda^n t^{n-1}}{(n-1)!} e^{-\lambda t} + \qquad (t \geq 0) +$$ + +for some $n \in \NN$ and $\lambda > 0$. + +The parameters $n$ and $\lambda$ are called the **shape** and **rate** +parameters respectively. + +The next figure shows the shape for two parameterizations. + + +```{code-cell} ipython3 +:tags: [hide-input] + +t_grid = np.linspace(0, 50, 100) + +class Erlang: + + def __init__(self, λ=0.5, n=10): + self.λ, self.n = λ, n + + def __call__(self, t): + n, λ = self.n, self.λ + return (λ**n * t**(n-1) * np.exp(-λ * t)) / factorial(n-1) + +e1 = Erlang(n=10, λ=0.5) +e2 = Erlang(n=10, λ=0.75) + +fig, ax = plt.subplots() +for e in e1, e2: + ax.plot(t_grid, e(t_grid), label=f'$n={e.n}, \lambda={e.λ}$') + +ax.legend() +plt.show() + +``` + +The CDF of the Erlang distribution is + +$$ + F(t) + = \PP\{W \leq t\} + = 1 - \sum_{k=0}^{n-1} \frac{(\lambda t)^k}{k!} e^{-\lambda t} +$$ (erlcdf) + +The Erlang distribution is of interest to us because of the following fact. + +```{proof:lemma} Distribution of Sum of Exponentials +:label: erlexp + +If, for some $\lambda > 0$, the sequence $(W_i)$ is IID and exponentially +distributed with rate $\lambda$, then $J_n := \sum_{i=1}^n W_i$ has the Erlang +distribution with shape $n$ and rate $\lambda$. +``` + +This connects to Poisson process theory, as we shall soon see. + + + + + +## Exercises + +### Exercise 1 + +Due to its memoryless property, we can "stop" and "restart" an exponential +draw without changing its distribution. + +To illustrate this, we can think of fixing $\lambda > 0$, drawing from +$\Exp(\lambda)$, and stopping and restarting whenever a threshold $s$ is crossed. + +In particular, consider the random variable $X$ defined as follows: + +* Draw $Y$ from $\Exp(\lambda)$. +* If $Y \leq s$, set $X = Y$. +* If not, draw $Z$ independently from $\Exp(\lambda)$ and set $X = s + Z$. + +Show that $X \sim \Exp(\lambda)$. + + +### Exercise 2 + +Fix $\lambda = 0.5$ and $s=1.0$. + +Simulate 1,000 draws of $X$ using the algorithm above. + +Plot the fraction of the sample exceeding $t$ for each $t \geq 0$ (on a grid) +and compare to $t \mapsto e^{-\lambda t}$. + +Is the fit good? How about if the number of draws is increased? + +Are the results in line with those of the previous exercise? + + +## Solutions + + +### Solution to Exercise 1 + +Let $X$ be constructed as in the statement of the exercise and fix $t > 0$. + +Notice that $X > s + t$ if and only if $Y > s$ and $Z > t$. + +As a result of this fact and independence, + +$$ + \PP\{X > s + t\} + = \PP\{Y > s \} \PP\{Z > t\} + = e^{-\lambda(s + t)} +$$ + +At the same time, + +$$ + \PP\{X > s - t\} + = \PP\{Y > s - t \} + = e^{-\lambda(s - t)} +$$ + +Either way, we have $X \sim \Exp(\lambda)$, as was to be shown. + + +### Solution to Exercise 2 + +Here's one solution, starting with 1,000 draws. + +```{code-cell} ipython3 + +λ = 0.5 +np.random.seed(1234) +t_grid = np.linspace(0, 10, 200) + +@njit +def draw_X(s=1.0, n=1_000): + draws = np.empty(n) + for i in range(n): + Y = np.random.exponential(scale=1/λ) + if Y <= s: + X = Y + else: + Z = np.random.exponential(scale=1/λ) + X = s + Z + draws[i] = X + return draws + +fig, ax = plt.subplots() +draws = draw_X() +empirical_exceedance = [np.mean(draws > t) for t in t_grid] +ax.plot(t_grid, np.exp(- λ * t_grid), label='exponential exceedance') +ax.plot(t_grid, empirical_exceedance, label='empirical exceedance') +ax.legend() + +plt.show() + +``` + +The fit is already very close, which matches with the theory in Exercise 1. + +The two lines become indistinguishable as $n$ is increased further. + +```{code-cell} ipython3 + +fig, ax = plt.subplots() +draws = draw_X(n=10_000) +empirical_exceedance = [np.mean(draws > t) for t in t_grid] +ax.plot(t_grid, np.exp(- λ * t_grid), label='exponential exceedance') +ax.plot(t_grid, empirical_exceedance, label='empirical exceedance') +ax.legend() + +plt.show() + +``` + + diff --git a/_sources/poisson.ipynb b/_sources/poisson.ipynb new file mode 100644 index 0000000..39358b5 --- /dev/null +++ b/_sources/poisson.ipynb @@ -0,0 +1,697 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Poisson Processes\n", + "\n", + "\n", + "## Overview\n", + "\n", + "Counting processes count the number of \"arrivals\" occurring by a given time\n", + "(e.g., the number of visitors to a website, the number of customers arriving at a restaurant, etc.)\n", + "\n", + "Counting processes become Poisson processes when the time interval between\n", + "arrivals is IID and exponentially distributed.\n", + "\n", + "Exponential distributions and Poisson processes have deep connections to\n", + "continuous time Markov chains.\n", + "\n", + "For example, Poisson processes are one of the simplest nontrivial examples of\n", + "a continuous time Markov chain.\n", + "\n", + "In addition, when continuous time Markov chains jump between states, the time\n", + "between jumps is *necessarily* exponentially distributed.\n", + "\n", + "In discussing Poisson processes, we will use the following imports:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "import quantecon as qe\n", + "from numba import njit\n", + "from scipy.special import factorial, binom" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Counting Processes\n", + "\n", + "Let's start with the general case of an arbitrary counting process.\n", + "\n", + "\n", + "### Jumps and Counts\n", + "\n", + "Let $(J_k)$ be an increasing sequence of nonnegative random variables\n", + "satisfying $J_k \\to \\infty$ with probability one.\n", + "\n", + "For example, $J_k$ might be the time the $k$-th customer arrives at a shop.\n", + "\n", + "Then \n", + "\n", + "$$\n", + " N_t := \\sum_{k \\geq 0} k \\mathbb{1} \\{ J_k \\leq t < J_{k+1} \\}\n", + "$$ (defcount)\n", + "\n", + "is the number of customers that have visited by time $t$.\n", + "\n", + "\n", + "\n", + "The next figure illustrate the definition of $N_t$ for a given jump sequence $\\{J_k\\}$." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "tags": [ + "hide-input" + ] + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEMCAYAAADtdfykAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAANvUlEQVR4nO3db2gkh3nH8d9Tee+0V9nVIRkaScG60iIEkS0V0RdxwUcbV8YtquoX6T8XCqWmpS30uBOc2kLAYJxyplyhgViVQ3NwtBRX1cW1YxEa5D8vGixHQWvfnSCkDrerw5GvlhuRPaOoT1/oD9J5c9pdze48I30/YJBmVzsPs8PXc7Ozu+buAgDE9VNpDwAAuDtCDQDBEWoACI5QA0BwhBoAgrunEQ/a2dnpvb29jXhoADiU3n777Q/c/f5KtzUk1L29vZqfn2/EQwPAoWRm3/9Jt3HqAwCCI9QAEByhBoDgCDUABEeoASC4fa/6MLNWSa9LOr51/xfd/QuNHgzAwcwslHRhdknLq2V1tec1PtKnsaHutMdCHaq5PO9jSb/i7mtmlpP0ppl93d3/q8GzAajTzEJJE9MFldc3JEml1bImpguSRKwzaN9Q++bnoK5t/Zrb+o/PRgUCuzC7tBPpbeX1DZ299JounplMaarDb25uriGPW9U5ajNrMbPvSPqBpG+4+7cq3OcpM5s3s/mVlZWk5wRQg+XVcsXlG8fua/IkSEJV70x09w1Jg2bWLunfzewz7v7OHfeZlDQpScPDwxxxAynqas+rVCHW3SdPNOyoD41T01Uf7r4qaU7SYw2ZBkAixkf6lM+17FmWz7VofKQvpYlwEPuG2szu3zqSlpnlJX1O0vVGDwagfmND3Xr2iQHdf6JFJqm7Pa9nnxjghcSMqubUx6ckfdXMWrQZ9n919/9o7FgADmpsqFu9tvl60eDgYMrT4CCquepjUdJQE2YBAFTAOxMBIDhCDQDBEWoACI5QA0BwhBoAgiPUABAcoQaA4Ag1AARHqAEgOEINAMERagAIjlADQHCEGgCCI9QAEByhBoDgCDUABEeoASA4Qg0AwRFqAAiOUANAcIQaAIIj1AAQHKEGgOAINQAER6gBIDhCDQDBEWoACI5QA0BwhBoAgiPUABAcoQaA4Ag1AARHqAEgOEINAMERagAIjlADQHCEGgCCI9QAEByhBoDgCDUABEeoASA4Qg0AwRFqAAiOUANAcIQaAIIj1AAQHKEGgOAINQAER6gBIDhCDQDBEWoACI5QA0BwhBoAgiPUABAcoQaA4Ag1AARHqAEguHv2u4OZfVrSJUk/K+n/JE26+983ejDgMJhZKOnC7JKWV8vqas9rfKRPY0PdaY+FjNk31JJ+LOmsu3/bzO6V9LaZfcPdrzZ4NiDTZhZKmpguqLy+IUkqrZY1MV2QJGKNmuwbane/Kenm1s8/NLNrkrolEWrgLi7MLu1Eelt5fUNnL72mi2cmmzLD2tqapqammrIuNE5N56jNrFfSkKRvVbjtKTObN7P5lZWVZKYDMmx5tVxx+cax+5o2Q1tbmwYHB5u2PjRGNac+JElm1ibp3yT9pbv/7523u/ukpElJGh4e9sQmBDKqqz2vUoVYd588obm5ueYPhMyq6ojazHLajPRld59u7EjA4TA+0qd8rmXPsnyuReMjfSlNhKyq5qoPk/SCpGvu/neNHwk4HLZfMHzmpYI++NEGV32gbtWc+nhY0h9IKpjZd7aW/ZW7v9K4sYDDYWyoW722+ZoN54pRr2qu+nhTkjVhFgBABbwzEQCCI9QAEByhBoDgCDUABEeoASA4Qg0AwRFqAAiOUANAcIQaAIIj1AAQHKEGgOAINQAER6gBIDhCDQDBEWoACI5QA0BwhBoAgiPUABAcoQaA4Ag1AARHqAEgOEINAMERagAIjlADQHCEGgCCI9QAEByhBoDgCDUABEeoASA4Qg0AwRFqAAiOUANAcIQaAIIj1AAQHKEGgOAINQAER6gBIDhCDQDBEWoACI5QA0BwhBoAgiPUABAcoQaA4Ag1AARHqAEgOEINAMERagAIjlADQHCEGgCCI9QAEByhBoDgCDUABEeoASA4Qg0AwRFqAAiOUANAcIQaAIK7Z787mNlXJP2GpB+4+2caPxLqNbNQ0oXZJS2vltXVntf4SJ/GhrrTHgvAAVVzRP1Pkh5r8Bw4oJmFkiamCyqtluWSSqtlTUwXNLNQSns0AAe07xG1u79uZr2NHwUHcWF2SeX1jT3LyusbOnvpNV08M5nSVJCktbU1TU1NpT0GMiyxc9Rm9pSZzZvZ/MrKSlIPiyotr5YrLt84dl+TJ8Gd2traNDg4mPYYyLB9j6ir5e6TkiYlaXh42JN6XFSnqz2vUoVYd588obm5ueYPBCAxXPVxSIyP9Cmfa9mzLJ9r0fhIX0oTAUhKYkfUSNf21R3PvFTQBz/a4KoP4BCp5vK8f5Z0WlKnmRUlfcHdX2j0YKjd2FC3em3z9QHOiQKHRzVXffxuMwYBAFTGOWoACI5QA0BwhBoAgiPUABAcoQaA4Ag1AARHqAEgOEINAMERagAIjlADQHCEGgCCI9QAEByhBoDgCDUABEeoASA4Qg0AwRFqAAiOUANAcIQaAIIj1AAQHKEGgOAINQAER6gBIDhCDQDBEWoACI5QA0BwhBoAgiPUABAcoQaA4Ag1AARHqAEgOEINAMERagAIjlADQHCEGgCCI9QAEByhBoDgCDUABEeoASA4Qg0AwRFqAAiOUANAcIQaAIK7J+0BAKBW6+vrKhaLun37dtqj1Ky1tVU9PT3K5XJV/w2hBpA5xWJR9957r3p7e2VmaY9TNXfXrVu3VCwWderUqar/jlMfADLn9u3b6ujoyFSkJcnM1NHRUfO/BAg1gEzKWqS31TM3oQaA4Ag1AARHqAEgOEINAHV6/vnnZWa6du3azrL+/n699957ia6HUANAnRYXFzU4OKiXX35ZkvTxxx/r/fff1wMPPJDoegg1ANSpUCjo/PnzO6F+99131d/fn/gVKbzhBUDmnT59OtHHm5ubq+p+V69e1ejoqJ5++ml99NFHKhQKGhgYkCR9+OGHOnnyZCLzcEQNAHW4ceOGOjo6lM/n9eijj2p2dlaLi4t68MEHJUlnzpxJbF0cUQPIvGqPgJO0uLi4c/T8+OOP6/Lly7p586bGxsb06quv6vr163ruued07ty5A6+LI2oAqMPu0xyPPPKI3njjjZ14d3Z26sknn0wk0lKVoTazx8xsycy+a2bnE1nzHWYWSnr4i9/UqfMv6+EvflMzC6VGrAYAErE71MePH9fAwICOHTum9vZ2LS4u6qGHHkpsXfue+jCzFklfkvSopKKkt8zsa+5+NakhZhZKmpguqLy+IUkqrZY1MV2QJI0NdSe1GgBIzOXLl/f8fuXKlZ2fOzs7NTU1pc7OTvX39x94XdWco/4lSd919+9Jkpn9i6TflJRYqC/MLu1Eelt5fUNnL72mi2cmk1rNkbC2tqapqam0xwCOtNHRUY2Ojib2eNWc+uiWdGPX78WtZXuY2VNmNm9m8ysrKzUNsbxarrh849h9NT0OpLa2Ng0ODqY9BoAEVXNEXenKbf/EAvdJSZOSNDw8/Inb76arPa9ShVh3nzyRyqu5ABBJNUfURUmf3vV7j6TlJIcYH+lTPteyZ1k+16Lxkb4kVwMAmVTNEfVbkn7BzE5JKkn6HUm/l+QQ2y8YXphd0vJqWV3teY2P9PFCIoCfyN0z+eUB7jWdcJBURajd/cdm9ueSZiW1SPqKu79b+3h3NzbUTZgBVKW1tVW3bt3K3NdxbX9nYmtra01/V9U7E939FUmv1DMYACStp6dHxWJRtV64EMH2t5DXgreQA8icXC5X07d4Zx1vIQeA4Ag1AARHqAEgOKvnUpF9H9RsRdL36/zzTkkfJDjOUcQ2jCfN54T9IRsecPf7K93QkFAfhJnNu/tw2nNkGdswnjSfE/aH7OPUBwAER6gBILiIoeZzTQ+ObRhPms8J+0PGhTtHDQDYK+IRNQBgF0INAMERagAIjlADQHAhQm1mf2ZmF9OeI+vYjvGk+ZywPxweIUIt6UFJi2kPcQiwHeNJ8zlhfzgkooR6QOxQSWA7xpPmc8L+cEikfh21bX6Pzv9I6nL3T34VOarCdownzeeE/eFwiXBEfUrS++5eNrOfNrOvmtk/mtnvpz1Yxuzejj9nZi+Y2YtpD3XE7X5Oxrb26ytm9mtNXne/mX3ZzF40sz9twrqRsAih3v3PsyckvejufyxpNL2RMmlnO7r799z9j1KeB3ufk5mt/foPJf12k9d9zd3/RNLnJfEpehkUIdS7X/DokXRj6+eNdMbJLF44iqfSc/I3kr7U7HWb2aikNyX9ZxPWjYRFCPUvSnpr6+eiNmMtxZgtS3ZvR8Sw85zYpr+V9HV3/3Yz1y1J7v41d/+sJE4pZlCq30JuZp+SNCTp9a1F05L+wcx+XdJLqQ2WMXduRzPrkPSMpCEzm3D3Z9Oc7yiqsG//haTPSfoZM/t5d/9ys9ZtZqe1eVrxuKRXGrVeNE5qV32Y2Tlt/t/9r92dnadObMd40nxO2B8Op9QvzwMA3B3ngQEgOEINAMERagAIjlADQHCEGgCCI9QAEByhBoDgCDWODDPrMbNmfCASkChCjaPkV7X5GRhApvDORBwJZvbLkq5IWpX0Q0m/5e7/ne5UQHUINY4MM3tV0jl3fyftWYBacOoDR0mfpKW0hwBqRahxJGx99OtH7r6e9ixArQg1jopTkpbTHgKoB6HGUXFdUqeZvWNmn017GKAWvJgIAMFxRA0AwRFqAAiOUANAcIQaAIIj1AAQHKEGgOAINQAE9/+sUEHDF/AHnAAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/poisson_3_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "Ks = 0, 1, 2, 3\n", + "Js = 0, 0.8, 1.8, 2.1, 3\n", + "n = len(Ks)\n", + "\n", + "fig, ax = plt.subplots()\n", + "\n", + "ax.plot(Js[:-1], Ks, 'o')\n", + "ax.hlines(Ks, Js[:-1], Js[1:], label='$N_t$')\n", + "ax.vlines(Js[:-1], (0, Ks[0], Ks[1], Ks[2]), Ks, alpha=0.25)\n", + "\n", + "ax.set(xticks=Js[:-1],\n", + " xticklabels=[f'$J_{k}$' for k in range(n)],\n", + " yticks=(0, 1, 2, 3),\n", + " xlabel='$t$')\n", + "\n", + "ax.legend(loc='lower right')\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "An alternative but equivalent definition is\n", + "\n", + "$$\n", + " N_t = \\max \\{k \\geq 0 \\,|\\, J_k \\leq t \\}\n", + "$$\n", + "\n", + "As a function of $t$, the process $N_t$ is called a **counting process**.\n", + "\n", + "The jump times $(J_k)$ are sometimes called **arrival times** and the\n", + "intervals $J_k - J_{k-1}$ are called **wait times** or **holding times**.\n", + "\n", + "\n", + "\n", + "\n", + "### Exponential Holding Times\n", + "\n", + "A Poisson process is a counting process with independent exponential holding times.\n", + "\n", + "In particular, suppose that the arrival times are given by $J_0 = 0$ and \n", + "\n", + "$$\n", + " J_k := W_1 + \\cdots W_k \n", + "$$\n", + "\n", + "where $(W_i)$ are IID exponential with some fixed rate $\\lambda$.\n", + "\n", + "Then the associated counting process $(N_t)$ is called a **Poisson process** with rate $\\lambda$.\n", + "\n", + "The rationale behind the name is that, for each $t > 0$, the random variable\n", + "$N_t$ has the Poisson distribution with parameter $t \\lambda$.\n", + "\n", + "In other words, \n", + "\n", + "$$ \n", + " \\PP\\{N_t = k\\} \n", + " = e^{-t \\lambda} \\frac{(t \\lambda)^k }{k!}\n", + " \\qquad (k = 0, 1, \\ldots)\n", + "$$ (poissondist)\n", + "\n", + "For example, since $N_t = 0$ if and only if $W_1 > t$, we have\n", + "\n", + "$$ \n", + " \\PP\\{N_t =0\\} \n", + " = \\PP\\{W_1 > t\\}\n", + " = e^{-t \\lambda}\n", + "$$\n", + "\n", + "and the right hand side agrees with {eq}`poissondist` when $k=0$.\n", + "\n", + "This sets up a proof by induction, which is time consuming but not difficult\n", + "--- the details can be found in $\\S29$ of {cite}`howard2017elements`.\n", + "\n", + "Another way to show that $N_t$ is Poisson with rate $\\lambda$ is appeal to \n", + "{proof:ref}`erlexp`.\n", + "\n", + "We observe that\n", + "\n", + "$$ \n", + " \\PP\\{N_t \\leq n\\} \n", + " = \\PP\\{J_{n+1} > t\\} \n", + " = 1 - \\PP\\{J_{n+1} \\leq t\\}\n", + "$$\n", + "\n", + "Inserting the expression for the Erlang CDF in {eq}`erlcdf` with shape $n+1$ and\n", + "rate $\\lambda$, we obtain \n", + "\n", + "$$ \n", + " \\PP\\{N_t \\leq n\\} \n", + " = \\sum_{k=0}^{n} \\frac{(t \\lambda )^k}{k!} e^{-t \\lambda}\n", + "$$\n", + "\n", + "This is the (integer valued) CDF for the Poisson distribution with parameter\n", + "$t \\lambda$.\n", + "\n", + "An exercise at the end of the lecture asks you to verify that $N(t)$ is Poisson-$(t \\lambda )$ informally via simulation.\n", + "\n", + "The next figure shows one realization of a Poisson process $(N_t)$, with jumps\n", + "at each new arrival." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "tags": [ + "hide-input" + ] + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD1CAYAAAB5n7/BAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAN+klEQVR4nO3db2hd933H8c83siIdIxsZybDpukSmDCGwIomJPlgexJQFhcykIjDotnQMRv1gbHiZo2HtSSEQEuzMbA8mHJGWecysbJ2mLE1mUchspWMtladMt7Zs1m0p0bVILC9yI3ItVN3vHlhy7ViyjuRzdb9Xfr8gIJ177jlf5887P517rq65uwAAcT1S6QEAAPdHqAEgOEINAMERagAIjlADQHA7ynHQ5uZmb21tLcehAWBbunDhwqy7713tsbKEurW1VePj4+U4NABsS2b207Ue49IHAARHqAEgOEINAMERagAIjlADQHCp7/owsxpJ45IK7n6ofCMB2KiRiYJOjF7R1bmiWhoT9fe2qa87V+mxkJGN3J53RNKUpN1lmgXAJoxMFDQwnFdxcUmSVJgramA4L0nEeptIFWoz2yfpNyS9LOlP1tu/VCqpWCw+4GgA0jh+dup2pFcUF5d09PR5nTxyqkJTPZxGR0fLcty016j/QtKfSiqttYOZHTazcTMbn52dzWQ4AOububGw6valOn743S7WXVGb2SFJH7v7BTM7uNZ+7j4kaUiSenp6PEmSzIYEsLaWxkSFuXt/gs3t2amxsbEKTISspVlRPyHpWTP7QNK3JX3ZzP62rFMBSK2/t01Jbc1d25LaGvX3tlVoImRt3VC7+4C773P3VklflfSuuz9f9skApNLXndMrz3Vo784amaRcY6JXnuvghcRtpCy/lAnA1urrzqnVrkmSurq6KjwNsrahULv7OUnnyjIJAGBVvDMRAIIj1AAQHKEGgOAINQAER6gBIDhCDQDBEWoACI5QA0BwhBoAgiPUABAcoQaA4Ag1AARHqAEgOEINAMERagAIjlADQHCEGgCCI9QAEByhBoDgCDUABEeoASA4Qg0AwRFqAAiOUANAcIQaAIIj1AAQHKEGgOAINQAER6gBIDhCDQDBEWoACI5QA0BwhBoAgiPUABAcoQaA4Ag1AARHqAEgOEINAMERagAIjlADQHCEGgCC27HeDmZWL2lMUt3y/t9x92+UezBUj5GJgk6MXtHVuaJaGhP197aprztX6bGAbWPdUEtakPRld583s1pJ3zezf3H3H5R5NlSBkYmCBobzKi4uSZIKc0UNDOcliVgDGVk31O7ukuaXv61d/svv95xSqaRisfjg0yG842enbkd6RXFxSUdPn9fJI6cqNNXDaX5+XoODg/y3tw2lukZtZjVm9r6kjyV9z91/uMo+h81s3MzGZ2dns54TQc3cWFh1+1Ld7i2eBA0NDers7Kz0GCiDNJc+5O5LkrrMrFHSP5nZAXf/8ef2GZI0JEk9PT2eJEnmwyKelsZEhbl7V3C5PTs1NjZWgYmA7WdDd324+5ykc5KeLss0qDr9vW1Kamvu2pbU1qi/t61CEwHbz7qhNrO9yytpmVki6dclXS73YKgOfd05vfJch/burJFJyjUmeuW5Dl5IBDKU5tLHL0s6bWY1uhX2v3f375Z3LFSTvu6cWu2aJKmrq6vC0wDbT5q7PiYldW/BLACAVfDORAAIjlADQHCEGgCCI9QAEByhBoDgCDUABEeoASA4Qg0AwRFqAAiOUANAcIQaAIIj1AAQHKEGgOAINQAER6gBIDhCDQDBEWoACI5QA0BwhBoAgiPUABAcoQaA4Ag1AARHqAEgOEINAMERagAIjlADQHCEGgCCI9QAEByhBoDgCDUABEeoASA4Qg0AwRFqAAiOUANAcIQaAIIj1AAQHKEGgOAINQAER6gBIDhCDQDB7VhvBzP7gqS/kfRLkkqShtz9L8s9GNY2MlHQidErujpXVEtjov7eNvV15yo9FoAyWTfUkn4u6ai7/4eZ7ZJ0wcy+5+6XyjwbVjEyUdDAcF7FxSVJUmGuqIHhvCQRa2CbWjfU7j4jaWb560/NbEpSTtKaoS6VSioWi5kNiV84fnbqdqRXFBeXdPT0eZ08cqpCU0nz8/MaHBzknztQBhu6Rm1mrZK6Jf1wlccOm9m4mY3Pzs5mMx3uMXNjYdXtS3W7t3iSuzU0NKizs7OiMwDbVZpLH5IkM2uQ9I+S/tjdf/b5x919SNKQJPX09HiSJJkNiV9oaUxUmLt31Zrbs1NjY2MVmAhAuaVaUZtZrW5F+oy7D5d3JNxPf2+bktqau7YltTXq722r0EQAyi3NXR8m6ZuSptz9ZPlHwv2svGD48lt5zX62xF0fwEMgzaWPJyR9TVLezN5f3vZn7v5O+cbC/fR159Rq1yRJXV1dFZ4GQLmluevj+5JsC2YBAKyCdyYCQHCEGgCCI9QAEByhBoDgCDUABEeoASA4Qg0AwRFqAAiOUANAcIQaAIIj1AAQHKEGgOAINQAER6gBIDhCDQDBEWoACI5QA0BwhBoAgiPUABAcoQaA4Ag1AARHqAEgOEINAMERagAIjlADQHCEGgCCI9QAEByhBoDgCDUABEeoASA4Qg0AwRFqAAiOUANAcIQaAIIj1AAQHKEGgOAINQAER6gBIDhCDQDBEWoACG7HejuY2bckHZL0sbsfKP9I6Y1MFHRi9IquzhXV0piov7dNfd25So8FAJlKs6L+a0lPl3mODRuZKGhgOK/CXFEuqTBX1MBwXiMThUqPBgCZWndF7e5jZta6kYOWSiUVi8XNzpTK8bNTKi4u3bWtuLiko6fP6+SRU2U9dwTz8/MaHBws+99nAJWX2TVqMztsZuNmNj47O5vVYdc0c2Nh1e1LdbvLfu4IGhoa1NnZWekxAGyBdVfUabn7kKQhSerp6fEkSbI69KpaGhMV5u5dTeb27NTY2FhZzw0AW6lq7/ro721TUltz17aktkb9vW0VmggAyiOzFfVWW7m74+W38pr9bIm7PgBsW2luz/s7SQclNZvZtKRvuPs3yz1YGn3dObXaNUlSV1dXhacBgPJIc9fHb23FIACA1VXtNWoAeFgQagAIjlADQHCEGgCCI9QAEByhBoDgCDUABEeoASA4Qg0AwRFqAAiOUANAcIQaAIIj1AAQHKEGgOAINQAER6gBIDhCDQDBEWoACI5QA0BwhBoAgiPUABAcoQaA4Ag1AARHqAEgOEINAMERagAIjlADQHA7Kj0AAGzU4uKipqendfPmzUqPsmH19fXat2+famtrUz+HUAOoOtPT09q1a5daW1tlZpUeJzV31/Xr1zU9Pa39+/enfh6XPgBUnZs3b6qpqamqIi1JZqampqYN/yRAqAFUpWqL9IrNzE2oASA4Qg0AwRFqAAiOUAPAJr3++usyM01NTd3e1t7erg8++CDT8xBqANikyclJdXV16e2335YkLSws6KOPPtJjjz2W6XkINQBsUj6f17Fjx26H+uLFi2pvb8/8jhTe8AKg6h08eDDT4507dy7VfpcuXdKzzz6rl156STdu3FA+n1dHR4ck6ZNPPtGePXsymYcVNQBswocffqimpiYlSaKnnnpKo6Ojmpyc1OOPPy5JeuGFFzI7FytqAFUv7Qo4S5OTk7dXz88884zOnDmjmZkZ9fX16ezZs7p8+bJee+01vfjiiw98LlbUALAJd17mePLJJ/Xee+/djndzc7Oef/75TCItpQy1mT1tZlfM7CdmdiyTM3/OyERBT7z6rvYfe1tPvPquRiYK5TgNAGTizlDX1dWpo6NDjz76qBobGzU5OanOzs7MzrXupQ8zq5H0V5KekjQt6Udm9s/ufimrIUYmChoYzqu4uCRJKswVNTCclyT1deeyOg0AZObMmTN3ff/mm2/e/rq5uVlvvPGGmpub1d7e/sDnSnON+kuSfuLu/yNJZvZtSV+RtGaoS6WSisVi6iGOn526HekVxcUlHT19XiePnLrvc+fn5zU4OLih8wGobu6uUqlU6THWdOjQIR06dEiSVp3T3TfUrDSXPnKSPrzj++nlbXcxs8NmNm5m47Ozs6kHkKSZGwurbl+q273ucxsaGjL9EQMAokmzol7tzm2/Z4P7kKQhSerp6fEkSVIP0dKYqDB37/9dcnt2amxsLPVxADwczEyPPFK990KYmTbSyDR/0mlJX7jj+32Srm5wrvvq721TUltz17aktkb9vW1ZngYAqlKaFfWPJP2Kme2XVJD0VUm/neUQKy8Ynhi9oqtzRbU0JurvbeOFRABrcveq/PAA93suSKxr3VC7+8/N7A8ljUqqkfQtd7+48fHur687R5gBpFJfX6/r169X3cdxrXxmYn19/Yael+qdie7+jqR3NjMYAGRt3759mp6e1rVr1yo9yoatfAr5RvAWcgBVp7a2dkOf4l3tqvdlUwB4SBBqAAiOUANAcLaZW0XWPajZNUk/zfzAALB9Pebue1d7oCyhBgBkh0sfABAcoQaA4Ag1AARHqFH1zKzRzP5g+esWM/tOpWcCssSLiah6ZtYq6bvufqDCowBlwVvIsR28KumLZva+pP+S1O7uB8zs9yT16dYvEzsg6c8lPSrpa5IWJD3j7v9nZl/UrY+b2yvpM0lfd/fLW//HAFbHpQ9sB8ck/be7d0nq/9xjB3Tr1/J+SdLLkj5z925J/y7pd5f3GZL0R+7+q5JelDS4JVMDKbGixnb3r+7+qaRPzeyGpLeWt+clPW5mDZJ+TdI/3PHrMuu2fkxgbYQa292dH8hZuuP7km79+/+IpLnl1TgQEpc+sB18KmnXZp7o7j+T9L9m9puSZLfwackIhVCj6rn7dUn/ZmY/lnRiE4f4HUm/b2b/KemipK9kOR/woLg9DwCCY0UNAMERagAIjlADQHCEGgCCI9QAEByhBoDgCDUABPf/nAVUuM3GMI4AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/poisson_5_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "np.random.seed(1234)\n", + "T = 5\n", + "Ws = np.random.exponential(size=T)\n", + "Js = np.cumsum(Ws)\n", + "Ys = np.arange(T)\n", + "\n", + "fig, ax = plt.subplots()\n", + "\n", + "ax.plot(np.insert(Js, 0, 0)[:-1], Ys, 'o')\n", + "ax.hlines(Ys, np.insert(Js, 0, 0)[:-1], Js, label='$N_t$')\n", + "ax.vlines(Js[:-1], Ys[:-1], Ys[1:], alpha=0.25)\n", + "\n", + "ax.set(xticks=[],\n", + " yticks=range(Ys.max()+1),\n", + " xlabel='time')\n", + "\n", + "ax.grid(lw=0.2)\n", + "ax.legend(loc='lower right')\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Stationary Independent Increments\n", + "\n", + "One of the defining features of a Poisson process is that it has stationary\n", + "and independent increments.\n", + "\n", + "This is due to the memoryless property of exponentials.\n", + "\n", + "It means in particular that\n", + "\n", + "1. the variables $\\{N_{t_{i+1}} - N_{t_i}\\}_{i \\in I}$ are independent for any\n", + " strictly increasing finite sequence $(t_i)_{i \\in I}$ and\n", + "2. the distribution of $N_{t+h} - N_t$ depends on $h$ but not $t$.\n", + "\n", + "\n", + "A detailed proof can be found in Theorem 2.4.3 of {cite}`norris1998markov`.\n", + "\n", + "Instead of repeating this, we provide some intuition from a discrete\n", + "approximation.\n", + "\n", + "In the discussion below, we use the following well known fact: If\n", + "$(\\theta_n)$ is a sequence such that $n \\theta_n$ converges, then\n", + "\n", + "$$\n", + " \\text{Binomial}(n, \\theta_n) \n", + " \\approx\n", + " \\text{Poisson}(n \\theta_n)\n", + " \\quad \\text{for large } n\n", + "$$ (binpois)\n", + "\n", + "(The exercises ask you to examine this claim visually.)\n", + "\n", + "We now return to {ref}`the environment ` where we linked the\n", + "geometric distribution to the exponential.\n", + "\n", + "That is, we fix small $h > 0$ and let $t_i := ih$ for all $i \\in \\ZZ_+$.\n", + "\n", + "Let $(V_i)$ be IID binary random variables with $\\PP\\{V_i = 1\\} = h \\lambda$ for some $\\lambda > 0$.\n", + "\n", + "Linking to our previous discussion, \n", + "\n", + "* either one or zero customers visits a shop at each $t_i$.\n", + "* $V_i = 1$ means that a customer visits at time $t_i$.\n", + "* Visits occur with probability $h \\lambda$, which is proportional to the\n", + " length of the interval between grid points.\n", + "\n", + "We learned that the wait time until the first visit is\n", + "approximately exponential with rate $t \\lambda$.\n", + "\n", + "Since $(V_i)$ is IID, the same is true for the second wait time and so on.\n", + "\n", + "Moreover, these wait times are independent, since they depend on separate\n", + "subsets of $(V_i)$.\n", + "\n", + "Let $\\hat N_t$ count the number of visits by time $t$, as shown in the next figure.\n", + "\n", + "($V_i = 1$ is indicated by a vertical line at $t_i = i h$.)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "tags": [ + "hide-input" + ] + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD9CAYAAACVzD2sAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAZ1UlEQVR4nO3db2yV93nG8esG/CeuDYZjwE4pMWpGBY0TKqy2Tv/EjRqapJiq8brWKJlQO5FKbdeljab0xVT6olSTsnZ/1DV4XedNi91spR3FpAlbUnftsGjtjeIAKasyQpwYbA7Y2HExDvz24hxaH2LiQ3Ke330l5/pISE4wPl/dj33z+PjhPBZCgIiI8JrnHSAiIq9Mi1pEhJwWtYgIOS1qERFyWtQiIuQWJPFBa2pqQn19fRIfWkTkDam/v/9UCGHpbL+XyKKur69HX19fEh9aROQNycyevdLv6akPERFyWtQiIuS0qEVEyGlRi4iQ06IWESGX96I2s/lm9j9m1p1kkIiI5LqaM+rPAziSVIiIiMwur+uozWwFgA8D+CqALyRaJCJz6tx/HLsOPO+dIZdZe+1CfLnl7QX/uPmeUf8lgD8FcPFK72BmW82sz8z6RkZGChJX7Pbt2+edQIVlHgwduw48j4HnznhnSCRznlGb2UYAwyGEfjNrvtL7hRDaAbQDQGNjo+5GUAB79+7FzTff7J1Bg2UeLB1VF8bwyL13emdIBPmcUb8HwCYzOwbguwBuNbN/TrRKAAAtLS3eCVRY5sHSUVOT8k6QSOZc1CGEL4UQVoQQ6gF8AsCTIYS7Ey8TvV7KZVjmwdIxPj7hnSCR6DpqYkNDQ94JVFjmwdIxNTXlnSCRXNWr54UQegD0JFIiL7N161bvBCos82DpuPbaa70TJBKdURNrb2/3TqDCMg+WjhdeeME7QSLRoia2evVq7wQqLPNg6aioqPBOkEi0qInpW9tcLPNg6SgtLfVOkEi0qIn19PR4J1BhmQdLx+joqHeCRKJFTay1tdU7gQrLPFg6li6d9fZ68gakRU2M5cyNBcs8WDp0Rl08tKiJpdNp7wQqLPNg6ZienvZOkEi0qImxXK/LgmUeLB0sP9SU5GlRE2O5XpcFyzxYOnQddfHQoibW0NDgnUCFZR4sHZWVld4JEokWNTF9IeZimQdLx/z5+vItFjrSxHp7e70TqLDMg6VjbOysd4JEokVNrK2tzTuBCss8WDqWL1/mnSCRaFET6+7WDd9nYpkHS0c6fdo7QSLRoiam1xvOxTIPlo6LF694C1N5g9GiJrZ582bvBCos82DpWLZMT30UCy1qYh0dHd4JVFjmwdJx4sQJ7wSJRIua2Pr1670TqLDMg6WjqqrKO0Ei0aIWESGnRU2sv7/fO4EKyzxYOsbHx70TJBItamJbtmzxTqDCMg+WjtraWu8EiUSLmlhnZ6d3AhWWebB0DA8PeydIJFrUxMrKyrwTqLDMg6Vj3jx9+RYLHWliGzdu9E6gwjIPlo5Uaol3gkSiRU2sq6vLO4EKyzxYOk6e1FMfxUKLmlhTU5N3AhWWebB0LFq00DtBItGiJjYxMeGdQIVlHiwdFy7otT6KhRY1sYGBAe8EKizzYOlg+QtDkqdFTYzlJqosWObB0qGb2xYPLWpiLDdRZcEyD5YO3dy2eGhRE0ulUt4JVFjmwdJRUlLinSCRaFETa25u9k6gwjIPlo7q6mrvBIlEi5rYzp07vROosMyDpWNkZMQ7QSLRoibGcubGgmUeLB06oy4eWtTE9MOiXCzzYOk4f/68d4JEokVN7OjRo94JVFjmwdIxOTnpnSCRaFETY7lelwXLPFg6dB118dCiJsZyvS4LlnmwdLA8BSPJ06ImVldX551AhWUeLB0sr4stydOiJtbY2OidQIVlHiwdVVWV3gkSiRY1sd27d3snUGGZB0vHqVNp7wSJZM5FbWblZvZzM/ulmR0ys6/ECBNgw4YN3glUWObB0rFkie7wUizyOaOeAnBrCOEmAOsA3G5m7042SwCey8BYsMyDpUOX5xWPBXO9QwghALj0wrcl2V8hySjJOHbsmHcCFYZ5dO4/jm/9qgw/2tHr2nF46Cwqps65Nkg8cy5qADCz+QD6AVwP4JshhP2zvM9WAFsBYOXKlYVsLFos1+uyYJjHrgPP48WSxd4ZWFu3ELes4rj6RJKX16IOIVwAsM7MqgH8wMxuCCE8ddn7tANoB4DGxkadcRdAe3s7tm3b5p1Bg2Ueb5o+g0fuvd07IzOLDTd6Z0gEV3XVRwhhFEAPAP/P0iJQX1/vnUCFZR7l5eXeCQB45iHJy+eqj6XZM2mY2TUAPgjg6aTDBFi9erV3AhWWeVRUVHgnAOCZhyQvnzPqOgA/NrODAH4B4N9DCN3JZgkA7N271zuBCss8Tp8+7Z0AgGcekrx8rvo4COAdEVrkMi0tLd4JVFjmUVPDcSsulnlI8vQvE4n19fV5J1Bhmcf4+MTc7xQByzwkeVrUxIaGhrwTqLDMY2pqyjsBAM88JHla1MQYrhtmwjIPlteBZpmHJE+LmhjL6x6zYJkHy+tAs8xDkqdFTUyXX+VimYcuz5PYtKiJsXyLzYJlHqWlpd4JAHjmIcnToibW09PjnUCFZR6jo6PeCQB45iHJ06Im1tra6p1AhWUeS5cu9U4AwDMPSZ4WNTGdMeVimYfOqCU2LWpi6bRutTQTyzymp6e9EwDwzEOSp0VNTNfJ5mKZB8sP8VjmIcnToiam62RzscxD11FLbFrUxBoaGrwTqLDMo7Ky0jsBAM88JHla1MRYFgILlnnMn8/xZcMyD0kex2eczKq31/cGqmxY5jE2dtY7AQDPPCR5WtTE2travBOosMxj+fJl3gkAeOYhydOiJtbdrRvpzMQyj3Sa4w4vLPOQ5GlRE2N53WMWLPO4ePGidwIAnnlI8rSoiW3evNk7gQrLPJYt43jqg2UekjwtamIdHR3eCVRY5nHixAnvBAA885DkaVETW79+vXcCFZZ5VFVVeScA4JmHJE+LWkSEnBY1sf7+fu8EKizzGB8f904AwDMPSZ4WNbEtW7Z4J1BhmUdtba13AgCeeUjytKiJdXZ2eidQYZnH8PCwdwIAnnlI8rSoiZWVlXknUGGZx7x5HF82LPOQ5HF8xsmsNm7c6J1AhWUeqdQS7wQAPPOQ5GlRE+vq6vJOoMIyj5MnOZ76YJmHJE+LmlhTU5N3AhWWeSxatNA7AQDPPCR5WtTEJiYmvBOosMzjwgWO1/pgmYckb4F3gFzZwMAAWltbvTNosMyDZUGyzMPD9PQ0BgcHce7cOe+Uq1ZeXo4VK1agpKQk7z+jRU1MNy/NxTIP3dzW3+DgIKqqqlBfXw8z887JWwgB6XQag4ODWLVqVd5/Tk99ENPNS3OxzEM3t/V37tw5pFKp19WSBgAzQyqVuurvBLSoiaVSKe8EKizzuJpvWZPEMg8vr7clfcmr6daiJtbc3OydQIVlHtXV1d4JAHjmIcnToia2c+dO7wQqLPMYGRnxTgDAMw9JnhY1MZ0x5WKZh86oZaaBgQHU1tbiqaeeSuwxtKiJsfzQigXLPM6fP++dAIBnHsVu+/bt2LdvH7Zv357YY+jyPGJHjx71TqDCMo/JyUnvBAA88yh2l/4pf5KvZqgzamLFfJ3sbFjmoeuoJbY5F7WZvcXMfmxmR8zskJl9PkaYFPd1srNhmQfLUw4s8yhmO3bsgJnhyJEjv/1/a9aswbFjxwr6OPmcUb8E4IshhDUA3g3gM2a2tqAVMqu6ujrvBCos82B5HWiWeRSzgwcPYt26ddizZw8AYGpqCidPnsR1111X0MeZ8znqEMIQgKHs2+NmdgTAmwEcLmiJvExjY6N3Ao3O/cfxw4lVeHJHr2vH4aGzWFlV6dpwiT4//A0MDOCBBx7AQw89hPvvvx+HDh3CmjVrCv6Pca7qOWozqwfwDgD7Z/m9rWbWZ2Z9LNeZvt7t3r3bO4HGrgPP4+jIb7wzsLZuIRae+V/vDAD6/GBw+PBhbNq0CcPDwxgbG8PAwAAaGhoK/jh5X/VhZpUAdgL4kxDC2ct/P4TQDqAdABobG0PBCovYhg0bvBOoXJ8qxyP3+r8G8759HJ/e+vzI+MruQzj8wstW0muy9tqF+HLL21/xfZ577jmkUilcc801uO222/D444/j4MGDuPHGGwEAZ86cweLFiwvSk9cZtZmVILOkHw4hfL8gjyxz0uVXuXRZXC6WjmJ18ODB354933nnndizZ0/OGfV9991XsMea84zaMk+2/D2AIyGErxfskWVOhf7J8esdy2sPsxwXlg5vc535JmXmUr7lllvw6U9/GpOTk2hoaMBjjz2Gp59+Gg8++CDuv//+1/xY+ZxRvwfAPQBuNbMD2V93vuZHljnpOtlcun45F0tHsZq5qMvKytDQ0IDS0lJUV1ejpqYGd999d0GWNJDHog4h/CyEYCGEG0MI67K/Hi3Io8sr0nWyuXT9ci6WjmL18MMP46677vrtf+/atQvHjx8HkHla5KabbirYY+lfJhKrr6/3TqBSXl7unQCA57iwdMjL1dTU4Nvf/nbOP4R5LfRaH8RWr17tnUCloqLCOwEAz3Fh6ZCX27RpEzZt2lSwj6czamJ79+71TqBy+vRp7wQAPMeFpUOSp0VNrKWlxTuBSk0Nx62nWI4LS4ckT4uaWF9fn3cClfHxCe8EADzHhaVDkqdFTWxoaMg7gcrU1JR3AgCe48LS4SUEjn8herVeTbcWNTFdJ5tL11HnYunwUF5ejnQ6/bpb1iEEpNPpq76CSVd9EGtvb8e2bdu8M2gwXUfNcFxYOjysWLECg4ODNDcavhrl5eVYsWLFVf0ZLWpiuvwqly7Py8XS4aGkpASrVq3yzohGT30QY/lWn0Vpaal3AgCe48LSIcnToibW09PjnUBldHTUOwEAz3Fh6ZDkaVETa21t9U6gsnTpUu8EADzHhaVDkqdFTUxnTLl0Rp2LpUOSp0VNLJ1OeydQmZ6e9k4AwHNcWDokeVrUxIr5OtnZsPzwjOW4sHRI8rSoien1hnMxXUfNgKVDkqdFTSyJuxm/nlVWVnonAOA5LiwdkjwtamIsi4nF/Pkcn64sx4WlQ5LH8Zkvs+rt7fVOoDI2dtY7AQDPcWHpkORpURNra2vzTqCyfPky7wQAPMeFpUOSp0VNrLu72zuBSjrNcYcXluPC0iHJ06ImxvL6yywuXrzonQCA57iwdEjytKiJbd682TuByrJlHE99sBwXlg5JnhY1sY6ODu8EKidOnPBOAMBzXFg6JHla1MTWr1/vnUClqqrKOwEAz3Fh6ZDkaVGLiJDToibW39/vnUBlfHzcOwEAz3Fh6ZDkaVET27Jli3cCldraWu8EADzHhaVDkqdFTayzs9M7gcrw8LB3AgCe48LSIcnToiZWVlbmnUBl3jyOT1eW48LSIcnj+MyXWW3cuNE7gUoqtcQ7AQDPcWHpkORpURPr6uryTqBy8iTHUx8sx4WlQ5KnRU2sqanJO4HKokULvRMA8BwXlg5JnhY1sYmJCe8EKhcucLzWB8txYemQ5GlRExsYGPBOoMKymFiOC0uHJE+LmphuXppLN7fNxdIhydOiJqabl+bSzW1zsXRI8rSoiaVSKe8EKiUlJd4JAHiOC0uHJE+Lmlhzc7N3ApXq6mrvBAA8x4WlQ5KnRU1s586d3glURkZGvBMA8BwXlg5J3pyL2sy+Y2bDZvZUjCD5HZ0x5dIZdS6WDklePmfUHQBuT7hDZsHywzMW58+f904AwHNcWDokeQvmeocQwn+aWX3yKTJT5/7j+KsBw64dvd4pFA4PnUXF1KR3BgDg6NGj3gkAeDokeXMu6nyZ2VYAWwFg5cqVhfqwRWvXgefxYsli7wwaa+sW4pZVdd4ZAHiuX2bpkOQVbFGHENoBtANAY2NjKNTHLWZvmj6DR+7Vs06XbNu2Ddhwo3cG2tvbMy3qkEh01Qcxvd5wrro6jjNqdUhsWtTEqqoqvROoNDY2eicAUIfEl8/leV0AegG8zcwGzexTyWcJAJw6lfZOoLJ7927vBADqkPjyueqjLUaIvNySJRx3NGGxYcMG7wQA6pD49NQHsclJjsvRWLBcjqYOiU2Lmti5c+e8E6gcO3bMOwGAOiQ+LWpiLK+/zILlumF1SGxa1MT0T4Rzsbz+sjokNi1qYuXl5d4JVOrr670TAKhD4tOiJlZRUeGdQGX16tXeCQDUIfFpURM7ffq0dwKVvXv3eicAUIfEp0VNrKZGt1qaqaWlxTsBgDokPi1qYuPjE94JVPr6+rwTAKhD4tOiJjY1NeWdQGVoaMg7AYA6JD4tamK6jjoXy3XD6pDYtKiJ6TrqXCzXDatDYtOiJqbL83KxXI6mDolNi5pYaWmpdwIVlqeC1CGxaVETGx0d9U6g0tPT450AQB0SnxY1saVLl3onUGltbfVOAKAOiU+LmpjOqHOxnEGqQ2LToiY2PT3tnUAlnea4NZk6JDYtamL6YVEuluuG1SGxaVET03XUuViuG1aHxKZFTayystI7gUpDQ4N3AgB1SHxa1MTmz9fhmYnlLy51SGzaBMTGxs56J1Dp7e31TgCgDolPi5rY8uXLvBOotLW1eScAUIfEp0VNLJ3WHV5m6u7u9k4AoA6JT4ua2MWLF70TqLC8Prc6JDYtamLLlumpj5k2b97snQBAHRKfFjWxEydOeCdQ6ejo8E4AoA6JT4uaWFVVlXcClfXr13snAFCHxKdFLSJCToua2Pj4uHcClf7+fu8EAOqQ+LSoidXW1nonUNmyZYt3AgB1SHxa1MSGh4e9E6h0dnZ6JwBQh8SnRU1s3jwdnpnKysq8EwCoQ+LTJiCWSi3xTqCyceNG7wQA6pD4tKiJnTyppz5m6urq8k4AoA6JT4ua2KJFC70TqDQ1NXknAFCHxKdFTezCBb3Wx0wTExPeCQDUIfFpURPTF2KugYEB7wQA6pD4tKiJ6ea2uVhu5qoOiU2LmphubpuL5Wau6pDY8lrUZna7mf3KzH5tZg8kHSUZJSUl3glUUqmUdwIAdUh8cy5qM5sP4JsA7gCwFkCbma1NOkyA6upq7wQqzc3N3gkA1CHxLcjjfd4J4NchhGcAwMy+C+AjAA4XOuYruw/h8Au6oSsAHB46i4op3Yprpp07d6KhocE7Qx0SXT6L+s0Anpvx34MA3nX5O5nZVgBbAWDlypUFiStma+sWYk2FfoQwE8sZpDokNgshvPI7mH0MwIdCCH+U/e97ALwzhPC5K/2ZxsbG0NfXV9BQEZE3MjPrDyE0zvZ7+ZyyDQJ4y4z/XgFAlyOIiESSz6L+BYDfM7NVZlYK4BMAfphsloiIXDLnc9QhhJfM7LMAHgcwH8B3QgiHEi8TEREA+f0wESGERwE8mnCLiIjMQpcViIiQ06IWESGnRS0iQk6LWkSE3Jz/4OVVfVCzEQDPvso/XgPgVAFzXi2GDoYGQB2XU0cuhg6GBuC1dVwXQlg6228ksqhfCzPru9K/zim2DoYGdajj9dDB0JBkh576EBEhp0UtIkKOcVGz3LaCoYOhAVDH5dSRi6GDoQFIqIPuOWoREcnFeEYtIiIzaFGLiJDTohYRIUexqM3seTNbpw51MDaoQx3eDe4/TDSzGgBDACpDCFPqUAdTgzrUwdDgekZtZtcjc+PceQDSZpY2s7xeI1sdb+wOhgZ1qIOlgeGM+jMA3h9C+Lg61MHWoA51MDQwPEd9E4ADl/7DzL5qZj81s++ZWYVHh5ktMrOfm9mEmd0QseHyjiYz6zWzn5hZl5mVOHUsN7N92Y4nzawudsMlZtaWfdGvmGbOot7MRsysJ/tr1hfRSboj29JsZk+Y2Y/N7KMeHWb23hmzOGpm33DqmGdmHdnd8TMzW+PQMN/MHs4ej+8U8uyaYVGvA/BLAMguxbeGEN4H4D8AfNKjA8AkgA8D+F7Ex5+t41kAt4YQbgHwDICPOHWcAvDebMc/AfiUQwPMbB6A30fm282YcjoA/CSE0Jz9FfMvjZlfK+UAvgjgjhDCB0IIP/DoCCH87NIsAOwD8G8eHdm3y7K740sA7nNo+CiAZ0IIHwDwNIC7CvUg3s9RzwNwA353lvA+AD/Kvv0jAO/16AghTEf+ArxSxwshhN9kf/slABedOi6EEC49dhWAxG9uPMvnBgBsRuYvzyhzeIWO92TP3LabmTl13AzgNwB2m9kPzKzWqePS/y8B8E4AP3XqGARwIXs8FiPCS57O0vDWGW//NzL7rCC8z6ivyf661LEYwFj27TEAS5w6vMzaYWarANwBoNurw8zWmdl+AJ9F5pMwaoOZzQfwBwAeifDYV+xA5if81wN4P4BlKOBZ01V2LM92tAD4OwDbnDouuQ3AEzP+Qo/dcQrAFDJnsn8D4G8dGg4DuDX79geR2WcF4bqYQggvAngIwGEzGwRwBsCi7G8vAnDaqcPFbB1mthDAPwK4J4Rw3qsjhHAghPAuAH+GzLeWsRvuBvAvERfBrB0hhKkQwosh81P47yPzHGX0DgCjAP4r+znxBIC1Th2XfAzAv8ZouELHBgDzQghvA9AK4C8cGroBnDOzJwG8CcDJQj4YzS8ADQA6s29vBfA5554OADc4Pv4CAHuQeZ7acw5lM97+EICvOzT8OYC9AB5D5rutv3aaxcIZb38NwB86daSQ+TmOAXgXgH9w/PwoAfAUMovSq+EOAA9m364H8JhXS7ZhGzJXgxTk47lfnnc5M/saMs9NDyPzRfCiU8ejyPyg4FkAO0IIHQ4N9wD4BjJfBADwrRBC7G/9YWZNyCzKCwDOAfhkCGEodseMHre7eZhZCzJfhJMA/g+ZWbzk1PIZAB9H5jn7T4YQnnHquAOZH2r+scfjZxsWAHgYQC2AMgBfCCHsi9xQC+C7yHydPBFC2F6wj822qEVEJJf3D89ERGQOWtQiIuS0qEVEyGlRi4iQ06IWESGnRS0iQk6LWkSEnBa1iAi5/weajo3pkQ/VYQAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/poisson_7_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "fig, ax = plt.subplots()\n", + "np.random.seed(1)\n", + "T = 10\n", + "p = 0.25\n", + "B = np.random.uniform(size=T) < p\n", + "N = np.cumsum(B)\n", + "m = N[-1] # max of N\n", + "\n", + "t_grid = np.arange(T)\n", + "t_ticks = [f'$t_{i}$' for i in t_grid]\n", + "ax.set_yticks(range(m+1))\n", + "ax.set_xticks(t_grid)\n", + "ax.set_xticklabels(t_ticks, fontsize=12)\n", + "\n", + "ax.step(t_grid, np.insert(N, 0, 0)[:-1], label='$\\hat N_t$')\n", + "\n", + "for i in t_grid:\n", + " if B[i]:\n", + " ax.vlines((i,), (0,), (m,), ls='--', lw=0.5)\n", + "\n", + "ax.legend(loc='center right')\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We expect from the discussion above that $(\\hat N_t)$ approximates a Poisson process.\n", + "\n", + "This intuition is correct because, fixing $t$, letting $k := \\max\\{i \\in\n", + "\\ZZ_+ \\,:\\, t_i \\leq t\\}$ and applying {eq}`binpois`, we have\n", + "\n", + "$$\n", + " \\hat N_t \n", + " = \\sum_{i=1}^k V_i\n", + " \\sim \\text{Binomial}(k, h \\lambda)\n", + " \\approx\n", + " \\text{Poisson}(k h \\lambda )\n", + "$$\n", + "\n", + "Using the fact that $kh = t_k \\approx t$ as $h \\to 0$, we see\n", + "that $\\hat N_t$ is approximately Poisson with rate $t \\lambda$, just as we\n", + "expected.\n", + "\n", + "\n", + "This approximate construction of a Poisson process helps illustrate the\n", + "property of stationary independent increments.\n", + "\n", + "For example, if we fix $s, t$, then $\\hat N_{s + t} - \\hat N_s$ is the number of visits\n", + "between $s$ and $s+t$, so that \n", + "\n", + "$$\n", + " \\hat N_{s+t} - \\hat N_s\n", + " = \\sum_i V_i \\mathbb 1\\{ s \\leq t_i < s + t \\}\n", + "$$\n", + "\n", + "Suppose there are $k$ grid points between $s$ and $s+t$, so that $t \\approx\n", + "kh$.\n", + "\n", + "Then\n", + "\n", + "$$\n", + " \\hat N_{s+t} - \\hat N_s\n", + " \\sim \\text{Binomial}(k, h \\lambda )\n", + " \\approx \n", + " \\text{Poisson}(k h \\lambda )\n", + " \\approx \n", + " \\text{Poisson}(t\\lambda)\n", + "$$\n", + "\n", + "This illustrates the idea that, for a Poisson process $(N_t)$, we have\n", + "\n", + "$$\n", + " N_{s+t} - N_s \n", + " \\sim \\text{Poisson}(t\\lambda)\n", + "$$\n", + "\n", + "In particular, increments are stationary (the distribution depends on $t$ but not $s$).\n", + "\n", + "The approximation also illustrates independence of increments, since, in the\n", + "approximation, increments depend on separate subsets of $(V_i)$.\n", + "\n", + "\n", + "\n", + "\n", + "## Uniqueness\n", + "\n", + "What other counting processes have stationary independent increments?\n", + "\n", + "Remarkably, the answer is none: \n", + "\n", + "```{proof:theorem} Characterization of Poisson Processes\n", + "\n", + "If $(M_t)$ is a stochastic process supported on $\\ZZ_+$ and starting at 0 with\n", + "the property that its increments are stationary and independent, then $(M_t)$ is a Poisson process.\n", + "\n", + "```\n", + "\n", + "In particular, there exists a $\\lambda > 0$ such that\n", + "\n", + "$$\n", + " M_{s + t} - M_s\n", + " \\sim \\text{Poisson}(t\\lambda)\n", + "$$\n", + "\n", + "for any $s, t$.\n", + "\n", + "The proof is similar to our earlier proof that the exponential distribution is\n", + "the only memoryless distribution.\n", + "\n", + "Details can be found in Section 6.2 of {cite}`pardoux2008markov` or \n", + "Theorem 2.4.3 of {cite}`norris1998markov`.\n", + "\n", + "(restart_prop)=\n", + "### The Restarting Property\n", + "\n", + "An important consequence of stationary independent increments is the\n", + "restarting property, which means that, when simulating, we can freely stop and\n", + "restart a Poisson process at any time:\n", + "\n", + "```{proof:theorem} Poisson Processes can be Pause and Restarted\n", + "If $(N_t)$ is a Poisson process, $s > 0$ and \n", + "$(M_t)$ is defined by $M_t = N_{s+t} - N_s$ for $t \\geq 0$, then $(M_t)$ is a \n", + "Poisson process independent of $(N_r)_{r \\leq s}$.\n", + "```\n", + "\n", + "```{proof:proof}\n", + "Independence of $(M_t)$ and $(N_r)_{r \\leq s}$ follows from indepenence of the\n", + "increments of $(N_t)$.\n", + "\n", + "In view of the uniqueness statement above, we can verify that $(M_t)$ is a\n", + "Poisson process by showing that $(M_t)$ starts at zero, takes values in \n", + "$\\ZZ_+$ and has stationary independent increments.\n", + "\n", + "It is clear that $(M_t)$ starts at zero and takes values in $\\ZZ_+$.\n", + "\n", + "In addition, if we take any $t < t'$, then\n", + "\n", + "$$\n", + " M_{t'} - M_t = N_{s+t'} - N_{s + t}\n", + " \\sim \\text{Poisson}((t' - t) \\lambda)\n", + "$$\n", + "\n", + "Hence $(M_t)$ has stationary increments and, \n", + "using the relation $M_{t'} - M_t = N_{s+t'} - N_{s + t}$ again,\n", + "the increments are independent as well.\n", + " \n", + "We conclude that $(N_{s+t} - N_s)_{t \\geq 0}$ is indeed a \n", + "Poisson process independent of $(N_r)_{r \\leq s}$.\n", + "```\n", + "\n", + "\n", + "\n", + "## Exercises\n", + "\n", + "Exercise 1\n", + "----------\n", + "\n", + "Fix $\\lambda > 0$ and draw $\\{W_i\\}$ as IID exponentials with rate $\\lambda$.\n", + "\n", + "Set $J_n := W_1 + \\cdots W_n$ with $J_0 = 0$ and\n", + " $N_t := \\sum_{n \\geq 0} n \\mathbb 1\\{ J_n \\leq t < J_{n+1} \\}$.\n", + "\n", + "Provide a visual test of the claim that $N_t$ is Poisson with parameter $t\n", + "\\lambda$.\n", + "\n", + "Do this by fixing $t = T$, generating many independent draws of $N_T$ and\n", + "comparing the empirical distribution of the sample with a Poisson\n", + "distribution with rate $T \\lambda$.\n", + "\n", + "Try first with $\\lambda = 0.5$ and $T=10$.\n", + "\n", + "Exercise 2\n", + "----------\n", + "\n", + "\n", + "In the lecture we used the fact that $\\Binomial(n, \\theta) \\approx \\Poisson(n \\theta)$ when $n$ is large and $\\theta$ is small.\n", + "\n", + "Investigate this relationship by plotting the distributions side by side.\n", + "\n", + "Experiment with different values of $n$ and $\\theta$.\n", + "\n", + "\n", + "## Solutions\n", + "\n", + "\n", + "### Solution to Exercise 1\n", + "\n", + "Here is one solution. \n", + "\n", + "The figure shows that the fit is already good with a modest sample size.\n", + "\n", + "Increasing the sample size will further improve the fit." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3xUVfrH8c8zKaSQAiQQUiAICIQiIIIIKCgIqDRdEFFsi+gqi+4qLur+XCwrrH1dXXvBsrq4SlOaYi8ogdAhEEJLIQklCSF95vz+mAFDSGACSe4k87xfr7wmc+beO98xeJ+55557jxhjUEop5X1sVgdQSillDS0ASinlpbQAKKWUl9ICoJRSXkoLgFJKeSlfqwPUREREhImPj7c6hlJKNShr1qw5YIyJrNzeoApAfHw8iYmJVsdQSqkGRUT2VNWuXUBKKeWltAAopZSX0gKglFJeSguAUkp5qQZ1Elgp5XkcDgdpaWkcPXrU6ihey8/Pj5YtWxIaGlqj9bQAKGtsmAcrH4W8NAiLhcsehh4TrE6lzsCBAwcQETp16oTNpp0K9c0YQ1FREenp6QA1KgL611L1b8M8WDwd8vYBxvm4eLqzXTU4ubm5tGrVSnf+FhERgoKCiImJITs7u0bruvUXE5ERIpIsIikiMrOK1zuLyM8iUiIi91Vo7yQi6yr85IvIPa7XZolIeoXXrqhRclVvFiSlM2DOV7Sb+TkD5nzFgqT0s9vgykegrOjEtrIi5xGBanDsdjt+fn5Wx/B6gYGBlJWV1Wid03YBiYgP8BIwDEgDVovIImPMlgqLHQKmA2MrrmuMSQZ6VthOOjC/wiLPGWOerlFiVa8WJKXzwKcbKSqzA5CeW8QDn24EYGyvGPc3lJ8JO1dCypeYvDSkikWqa1eeT0T/clY7k7+BO+cA+gIpxphU15t8BIwBjhcAY0w2kC0iV55iO5cBO40xVV6RpjzTU8uTj+/8jykqszPzkw0s2ZhZ7Xq+ppT2xZvpWriarkWriStNBSDXpwV+pgnBUnLSOllEEFW78ZVSp+BOAYgB9lV4ngb0O4P3mgh8WKltmojcCCQC9xpjDp/BdlUdysgtqrK9uNzB3kOFJ7RF2ffTp3wNfcrWcl75BgIppgxfNvsmsDzgZtb49WaXLZ5zc5Yxx+8NgqT0+LpFxo/ZZeP5Z51+GqXc07RpUzZs2MA555xjdZQ65U4BqPJovSZvIiL+wGjggQrNLwOPubb1GPAMcGsV604FpgK0adOmJm+rakF0eCDn53/B/b7ziJYDZJgIniyfwJrQYSy783zY/QOkfAkpKyF3p3Ol8LbQ8QboMBS/+EH0bNKUnsAU1zYHzLEzM5/j2wTYY1qRGDLUks+oVGUFBQVWR6gX7hSANCCuwvNYIKOG7zMSWGuMyTrWUPF3EXkd+KyqFY0xrwGvAfTp00cnMK5nzydsp+ua376tx8oBnvF7lUK/FfCPvWAvBd9AaDcI+t0OHYZC83PgFP2RM4Z34oFPS1lUOhCAST4recLvTV7qtA5nT6HyBguS0nlqeTIZuUVEhwcyY3inmp1XUmfNnVFAq4GOItLO9U1+IrCohu9zHZW6f0SkdYWn44BNNdymqgfdtv3zhK4aAD+xE3Z0N/SdCpMXwF92w/UfOwtAi/an3PmD8+Tx7Ku7ExMeiAD/sV/KOv/e9Nz6NBxKrbPPojzHscEF6blFGH4bXHDWI8wqiY+PZ/bs2SQkJNCsWTNuueUWiouLAXj99dfp0KEDzZs3Z/To0WRk/Pa9VkRISUkBYMmSJSQkJBASEkJMTAxPP+0ct3LgwAGuuuoqwsPDad68OYMGDcLhcACwdetWBg8eTHh4OF27dmXRot92mTfffDN33XUXV155JSEhIfTr14+dO3fW6ud2lxhz+i/VriGazwM+wFvGmL+LyB0AxphXRCQKZz9+KOAACoAEY0y+iAThPIdwjjEmr8I238M5QsgAu4HbjTHVn1XEeQSgt4OuPw6HgUebYauyx09gVm6tvM+zK5L5+Ktf+D7kQXxbd4ebPwObT61sW9W9rVu30qVLFwAeWbyZLRn5p10naW8upXbHSe3+PjZ6tQk/5boJ0aH8bVRXt7LFx8fTtGlTli5dSnBwMKNGjWLIkCFceumlTJgwgRUrVtC1a1fuu+8+1q9fz3fffQc4C8COHTvo0KEDrVu3Zt68eQwaNIjDhw+za9cuevfuzQMPPMDhw4f517/+BcCqVasYOHAg5eXldOnShVtvvZX77ruPH374gTFjxpCYmEinTp24+eabWbRoEcuWLaN3797cdNNN2O12PvroI7c+06lU/FtUJCJrjDF9Kre7dR2AMWaJMeZcY0x7Y8zfXW2vGGNecf2+3xgTa4wJNcaEu37Pd71WaIxpUXHn72qfbIzpbozpYYwZfbqdv6p/yzbvJ8PRouoXw2Jr7X3uHNKBJi3ieNp2K+z9CVa9XGvbVp6pqp3/qdrPxrRp04iLi6N58+Y89NBDfPjhh3zwwQfceuut9O7dmyZNmjB79mx+/vlndu/efdL6fn5+bNmyhfz8fJo1a0bv3r2Pt2dmZrJnzx78/PwYNGgQIsKqVasoKChg5syZ+Pv7c+mll3LVVVfx4Ye/dYJcffXV9O3bF19fX66//nrWrVtX65/bHXorCFUlu8Pw7BfbGRY4iftL/oVUPArwC3TeuqGWBPj58PjY7tzw5lHGR19M+5WPQsdhENmp1t5D1Q93v5kPmPMV6VWMMIsJD+S/t/ev1Uxxcb+dwmzbti0ZGRlkZGQc35GDc9RPixYtSE9Pp/Ksg5988gmPP/44M2fOpEePHsyZM4f+/fszY8YMZs2axeWXXw7A1KlTmTlzJhkZGcTFxZ1wZXTbtm2P36oBICrqtwHPQUFBlp101mu3VZUWrksnJbuA0R2bOHf+wZGAQFgcjHqh1u/bM7BjBGN6xjA5axJ2vyCYfwfYy2v1PZTnmDG8E4F+J3bzBfr5MGN47Rf9fft+G8W+d+9eoqOjiY6OZs+e3y5JOnr0KAcPHiQm5uST0BdccAELFy4kOzubsWPHMmGC899+SEgIzzzzDKmpqSxevJhnn32WlStXEh0dzb59+46fDzj2vlVt22paANRJyuwOnv9yB31b2eic8ppzZM+MFGef/5821dlN2/56ZQJH/JrzYuAfIGMt/Ph8nbyPsl7lgQAx4YHMvrp7nYwCeumll0hLS+PQoUM88cQTXHvttUyaNIm3336bdevWUVJSwoMPPki/fv1O+vZfWlrKBx98QF5eHn5+foSGhuLj4yxcn332GSkpKRhjjrf7+PjQr18/goODefLJJykrK+Obb75h8eLFTJw4sdY/29nSLiB1ko8T09h7qJD3en6FbMuHoY/Uy/tGhjThLyM689cF5Vx9zkjivpkD5w6HqO718v6qfo3tFVMvwz4nTZrE5ZdfTkZGBmPGjOGvf/0rQUFBPPbYY1xzzTUcPnyYiy66qNqTsO+99x7Tpk3DbrfTqVMn3n//fQB27NjBtGnTyMnJoVmzZtx5550MHjwYgEWLFnHnnXcye/ZsYmJiePfdd+ncuXOdf9aacmsUkKfQUUB1r7jMzpCnv6Fb03xey70d6XY1jHul3t7f4TBc88pP5B3I4svA+7E1jYLbvgJf/3rLoGqmupEnniA+Pp433niDoUO94yLDOhkFpLzHf37ZS2ZeMX8PW+S8BHzIQ/X6/jab8MS47uwpDuC9iD9D1kb47sl6zaCUt9ACoI4rLC3n39+kcG1cHi1T50O/qRAed/oVa1mX1qH8fmA7/pbclgPtr4Hvn4X0NfWeQ6nGTguAOu6dn3ZzoKCUB5vMg4BQGPhny7LcM7QjMeGB3JZzDaZpK5j/BygrtiyPaph2797tNd0/Z0ILgAIgr6iMV79N5Y62GYSlfe3c+Qc1tyxPkL8vj4zuSlI2fNbuQTiQDF8/blkepRojLQAKgDd/2EVeUSnTHe9BaIzzvj4WG5rQiuFdWzEjKYKC7jfCTy/Cnp+tjqVUo6EFQHHoaClvfp/KQ22TCcpZ7zzx6xdodSwAZo3uio8I9+b+DhPeBhb8AUqPWh1LqUZBC4Di1W93UlpWwk1Fc6FlApznORestA4L5M+Xd2L5jgJWdX8MDu+GL/5mdSylGgUtAF4uO7+YuT/vZnabtfjn74GhszzuTpw39W9L1+hQ7l4VTEmfqbD6dUj9xupYSjV4WgC83Etfp+BvL2Rc/nvQdiB0vNzqSCfx9bHxxLju5BSU8GTZtdCiIyycBsV5p19ZqXqwd+9emjZtit1ur3aZkSNHMnfu3LN6n3feeYeBAwee1TYq0gLgxdIOF/KfX/fyXOx3+BQdhGGPnnYyF6ucFxfOjRe25a1f97PjoqcgPx2WP2h1LKUA53S1BQUFx+8TVJWlS5dy00031WOq09MC4MX+tTKFSPIYcngeJIyF2POtjnRK9w7vRGTTJtzzoy+Oi+6GpPdh+3KrY6kztWEePNcNZoU7HzfMszpRnTDGnHBnUE+iBcBL7TpwlP+tTeOfrZdjs5fW6v3960pogB9/G9WVzRn5zG1yHbTsCov+CIWHrI6mamrDPFg8HfL2Acb5uHh6nRSBjIwMrrnmGiIjI2nXrh0vvPACALNmzWL8+PHccMMNhISE0L17d7Zv387s2bNp2bIlcXFxrFix4vh2Bg8ezAMPPEDfvn0JCwtjzJgxHDrk/Le3e/duRITy8vLjyz700EMMGDCAoKAgUlNTGTx4MG+88cbx7b3++ut06dKFkJAQEhISWLt2LQBz5syhffv2x9vnz59f6/9NjtG7gXqp57/czrk+++lzcBH0ucU5l28DcEX3KAZ3iuTpL3dx1eTnifxwJCy9H6554/Qrq7q3dCbs33j65dJWg73kxLayIue5nTWn6SeP6g4j57gVx+FwMGrUKMaMGcOHH35IWloaQ4cOpVMn57wDixcvZuHChbzzzjvceuutDB8+nClTppCens4777zD7bffzq5du45v791332X58uW0a9eOG2+8kenTpx+/O2hl7733HkuXLqVTp05Uvunmxx9/zKxZs1iwYAF9+vRh586d+Pn5AdC+fXu+//57oqKi+Pjjj7nhhhtISUmhdevWVb3NWdEjAC+UvP8Ii9Zn8FzEIsQvEC75i9WR3CYiPDamG3Zj+OsqG1x8P2z8GLYstDqaqonKO//TtZ+h1atXk5OTw8MPP4y/vz/nnHMOt9122/FbPw8aNIjhw4fj6+vL+PHjycnJYebMmfj5+TFx4kR2795Nbu5vc19PnjyZbt26ERwczGOPPca8efOqPfF7880307VrV3x9fY/v3I954403uP/++7ngggsQETp06EDbtm0BGD9+PNHR0dhsNq699lo6duzIr7/+Wqv/XY5x6whAREYA/8Q5Kfwbxpg5lV7vDLwN9AYeMsY8XeG13cARwA6UH7slqYg0B/4LxOOcFH6CMebw2X0c5Y5nv0jmIv9UOh/+GgY/AE1bWh2pRuKaBzH9so48uSyZlb2u57LWS5z3Clo2E/IznfMVX/ZwnU1co07BzW/mPNfN1f1TSVgc3PJ5rcXZs2cPGRkZhIf/NtG83W5n0KBBtG3bllatWh1vDwwMJCIi4viJ3MBA58WQBQUFx9evPL1kWVkZBw4cqPK9Ky5b2b59+2jfvuqj7nfffZdnn332+PzEBQUF1b7H2TrtEYCI+AAvASOBBOA6EUmotNghYDrwNFUbYozpWel+1DOBlcaYjsBK13NVxzam5bF8837+EfYJBLeE/tOsjnRGbht0Due2asrDn22ntPMYKDsK+RnUdX+yqiWXPXzy1ea1PNc0OHfC7dq1Izc39/jPkSNHWLJkyRltr/L0kn5+fkRERFS5rJxiRF1cXBw7d+48qX3Pnj3cdtttvPjiixw8eJDc3Fy6det2UhdSbXGnC6gvkGKMSTXGlAIfAWMqLmCMyTbGrAbKavDeY4BjnX1zgbE1WFedoadXJDMqcCOx+Ukw+C/QpKnVkc6In+vagPTcIop+fPXkBcqKYOWj9R9MuafHBOfc0mFx1OVc03379iU0NJR//OMfFBUVYbfb2bRpE6tXrz6j7b3//vts2bKFwsJCHn74YX73u9+dcuhndaZMmcLTTz/NmjVrMMaQkpLCnj17OHr0KCJCZGQkAG+//TabNm06o6zucKcAxAAVj9XSXG3uMsAKEVkjIlMrtLcyxmQCuB4bVj9EA7R69yG+357FI0EfQ/P20NuzxiTXVJ/45ky8II6Qkv1VL5CXVr+BVM30mOCcY7oO55r28fFh8eLFrFu3jnbt2hEREcGUKVPIyzuziwgnT57MzTffTFRUFMXFxcdHFNXU+PHjeeihh5g0aRIhISGMHTuWQ4cOkZCQwL333kv//v1p1aoVGzduZMCAAWf0Hu447ZSQIjIeGG6MmeJ6Phnoa4z5YxXLzgIKKp0DiDbGZIhIS+AL4I/GmO9EJNcYE15hucPGmGZVbHMqMBWgTZs25+/Zs+dMPqfXM8Yw8bVVdM1axMOOf8P4udC14R905RaWUvhkF6Kpoo80LM65Y1F1ypOnhKxNgwcP5oYbbmDKlClWR6lWXUwJmQZUPJsRC2S4G8gYk+F6zAbm4+xSAsgSkdaucK2B7GrWf80Y08cY0+fYYZGquR9TDrJ+Vyb3+v0PYvpAwpjTr9QAhAf5k9Z7BoXmxDmDy30CGsS1DUpZyZ0CsBroKCLtRMQfmAgscmfjIhIsIiHHfgcuB459JVsEHOuDuAnQcXx1xBjDUyuSmR68kuCSbI++5cOZSI+9igfLp5DmiMAYMAZml01kgb3uDp2VagxOOwzUGFMuItOA5TiHgb5ljNksIne4Xn9FRKKARCAUcIjIPThHDEUA811nw32B/xhjlrk2PQeYJyK/B/YC42v3o6ljVm7NZs++fUxpuhDOHQHxjWvH+PSK7aTbB7LAPpBYyeE7/3sIdeTx1PJkxvaqyekqpar3zTffWB2h1rl1HYAxZgmwpFLbKxV+34+za6iyfOC8arZ5ELjM7aTqjDgchqdXJPNA08/xsx+FyxrfvfQzcouO/55mIvnK0ZPrfL7mpdxxFqZSyvPplcCN3JJNmRRkpfI7x1LkvEnQqvIlHA1fdPiJ48nftw+jpeRybcgGixJ5n7oap67cdyY3nNMC0IiV2x08+8V2ZjWdj83mA0Ma5+2TZwzvRKDfb2Oxv3X0YJ9pyd2h31qYynsEBARw8OBBLQIWMcZQWlpKeno6wcHBNVpXbwbXCC1ISuep5cmk5xaRILu5rMm3yIC7Iaxx9ocf6+c/9pkNNja1voaR+1+G7K3QsvEPUbRSbGwsaWlp5OTkWB3Fa/n6+hIWFlbtVcnVrldHeZRFFiSl88CnGykqc96g6i++H5FngvgxbCJXWpytLo3tFcPYXjEYYxj23He8WzyQET5vIavfhCuru0OJqg1+fn60a9fO6hjqDGgXUCPz1PJkhtm/5Qf/6aQ2mcQlPhv42n4eT3yVaXW0eiEi3DIgnp/3Cwfjr4T1H0FJgdWxlPJIWgAamT75XzDH7w1ibQewuYb6j/BJpE/+F9YGq0dX94olLNCPt0ouhdIjsFFvCqdUVbQANDIP+H9MkJSe0BYopTzg/7FFiepfoL8PE/vG8crOZpRGdoPVbzqvDlNKnUALQCPTqqp74pyivbG6sX88Ija+CB4FWZtg3y9WR1LK42gBaGQkrKrr8apvb6xiwgMZ0TWKR3d3wTQJhdU6ZaRSlWkBaGQOXjiTUlPp/uR1MNFGQ3DLgHiyin3ZFjUKNi+AAh2mqFRFWgAamfnlF5FqojA2X+pyoo2G4Py2zegeE8acAxeBowyS3rU6klIeRQtAI/PDhh10tGUiF02v04k2GgIR4daB8Xx7sBmHW10EiW+Do+oJvJXyRloAGpHs/GJaZHyFDw7ofJXVcTzCld2jiQxpwnv2oc65gnessDqSUh5DC0AjsnxLFsNsaygLjoLoXlbH8Qj+vjZu6NeWf6Z1pDw4Sk8GK1WBFoBG5KsNuxjssx7fLleBTf+0x1x/YRt8fPz4NuRKSPkSDqVaHUkpj6B7iUbi8NFS/Pd8RwClSBft/qkoomkTRveM5pH0Ps6T44lvWR1JKY+gBaCR+GJLFsNsq7H7h0L8QKvjeJxbBsSztyyM1IghkPQ+lBWdfiWlGjktAI3E8o1pDPNZi63zCPDxszqOx+kaHUa/ds15/vAgKDoMm+dbHUkpy2kBaATyi8soTf2RMAqQzqOsjuOxbhnQjsVH2lMQco7z/kBKeTm3CoCIjBCRZBFJEZGZVbzeWUR+FpESEbmvQnuciHwtIltFZLOI3F3htVkiki4i61w/V9TOR/I+X2/L5lJ+xeHTBDroNMvVGZbQithmQcxjOKQnQkaS1ZGUstRpC4CI+AAvASOBBOA6Eak8sewhYDpQeeaNcuBeY0wX4ELgrkrrPmeM6en6WYI6I0s3ZDLSdw3S/lLwr9mUcN7Exybc1D+e53LOx+EbqEcByuu5cwTQF0gxxqQaY0qBj4AxFRcwxmQbY1YDZZXaM40xa12/HwG2Ao1zXkKLFJaWk7PjF6I4gHTR7p/TmXBBHHb/EH5pOhQ2/s95PkApL+VOAYgB9lV4nsYZ7MRFJB7oBVS8L+80EdkgIm+JSLNq1psqIokikqhzjp7s2+QcLjG/YsQG546wOo7HCwv043fnxzInZwCUF8G6D62OpJRl3CkAUkVbjWbXEJGmwCfAPcaYfFfzy0B7oCeQCTxT1brGmNeMMX2MMX0iIyNr8rZeYemm/VzhuwbT5iIIbmF1nAbhpoviWW9vQ0ZID+eVwQ6H1ZGUsoQ7BSANiKvwPBbIcPcNRMQP587/A2PMp8fajTFZxhi7McYBvI6zq0nVQEm5nZRt6+nAPmx68Zfb2kc2ZUinSP59dDAc2gm7vrU6klKWcKcArAY6ikg7EfEHJgKL3Nm4iAjwJrDVGPNspddaV3g6DtjkXmR1zI8pBxhYvsr5pPOV1oZpYG4Z0I55hedT4t9M7w+kvJbv6RYwxpSLyDRgOeADvGWM2Swid7hef0VEooBEIBRwiMg9OEcM9QAmAxtFZJ1rkw+6Rvw8KSI9cXYn7QZur92P1vgt3bif6/3W4IjqgS28jdVxGpRBHSNo07IZC0svY3zyp0heOoTp+ATlXU5bAABcO+wlldpeqfD7fpxdQ5X9QNXnEDDGTHY/pqqszO4gacs2/sEObF0etDpOgyMi3HxRPC8sHMT4Jp/Amnfg0oesjqVUvdIrgRuoX1IP0bf0F2wYvff/Gbq6dwxHAqLZGNQP1s6F8lKrIylVr7QANFDLNmcywncNjmbtoGUXq+M0SEH+vkzsG8dzeRdDQRZs+8zqSErVKy0ADZDDYfhhUyoXySbn6B+pspdNueHG/vF8b87jcJNovTJYeR0tAA3Qmr2H6VH4K76Ua/fPWYoJD2R412jeKRkCe36A7G1WR1Kq3mgBaICWbtzPCN9EHMEtIVYvnzhbtwyI593iQdht/pCoRwHKe2gBaGCMMXy9aS+X+qzH1vkKnfqxFpzfthmxMXGstF2EWfchlBRYHUmpeqF7jwZmQ1oebY8kEmCKtPunlogItw6M5+WjQ5DSI7BxntWRlKoXWgAamGWb9zPSJxHj3xTaXWx1nEbjyu7RpAV3ZbdfB+fJYFOj210p1SBpAWhAjDGs2JjOCL8kpOPl4NvE6kiNhr+vjRsujOflwiGQtQn2/XL6lZRq4LQANCDJWUdodmgdYY5c0Ju/1bpJ/dqwjAEU2Zrq/YGUV9AC0IAs3bifET6JGB9/6DDM6jiNTmRIE4b1PIePywdhNi+AAp1/QjVuWgAakGUbM7mqSRLS7hIICLU6TqN0y4B43im7DHGUQdK7VsdRqk5pAWggUnMKkJwtRNkztfunDnWNDiMivhs7icN89XeYFQ7PdYMNOjJINT5aABqIZZv3c7ktEYNApyusjtOoPRS3iViTiRg7YCBvHyyerkVANTpaABqIZZv2MzZwLRLXD5q2tDpOo9Yj+Z80kfITG8uKYOWj1gRSqo5oAWgA0g4Xcig9hXPKU3Xmr/qQl1Zls6mmXamGSgtAA7Bsk7P7B9D+/3qQRUSN2pVqqLQANADLNu1nbMBaaNkVmp9jdZxGb3bpeAqN/wltRcaf2aXjLUqkVN3QAuDhsvOL2bV3D93sW7X7p54khg5jZtkU0hwROFx3hFhh701iqF57oRoXtwqAiIwQkWQRSRGRmVW83llEfhaREhG5z511RaS5iHwhIjtcj83O/uM0Psu3ZHGpbS02HNr9U09mDO/EFz6XMLD0Bc4p+Q8/2LvSz2c79w/Toy/VuJy2AIiID/ASMBJIAK4TkYRKix0CpgNP12DdmcBKY0xHYKXruapk2aZMxgUkYcLiIKqH1XG8wtheMcy+ujsx4YEI8Kb9CqLkEGP8E62OplStcucIoC+QYoxJNcaUAh8BYyouYIzJNsasBspqsO4YYK7r97nA2DP8DI3W4aOlbEjNoK9jPdJZp36sT2N7xfDjzEvZNedKyttdxm6icfz0kt4lVDUq7hSAGGBfhedprjZ3nGrdVsaYTADXY5WD20VkqogkikhiTo533Zvliy1ZDGA9vqZUu38sdPvgjrxRNhxb5lq9S6hqVNwpAFV97XT3a9DZrOtc2JjXjDF9jDF9IiMja7Jqg7ds837GBa7FBDaHuAutjuO1BnRowdZWV5JPU8zPL1kdR6la404BSAPiKjyPBTLc3P6p1s0SkdYArsdsN7fpFY4Ul/HLjv1cwlqk0xXg42t1JK8lItx8SVfeL78Utn4Gh3dbHUmpWuFOAVgNdBSRdiLiD0wEFrm5/VOtuwi4yfX7TcBC92M3fl9ty6aX2UyAvUCHf3qAkd2iWNl0NHYE88urVsdRqlactgAYY8qBacByYCswzxizWUTuEJE7AEQkSkTSgD8DfxWRNBEJrW5d16bnAMNEZAcwzPVcuSzd6Lz4y/gFQfshVsfxer4+NsYO7svn9n7Y17wLxflWR1LqrLnVr2CMWQIsqdT2SoXf9+Ps3nFrXVf7QeCymoT1FoWl5Xy7fT9PBgc0ViEAACAASURBVKxBOlwGfoFWR1LA+PNjuW3FaMaU/QTrPoAL/2B1JKXOil4J7IG+Tc6hU3kKoWUHoPMoq+MolwA/H/oOGMpqx7mU/vhvcNitjqTUWdEC4IGWbd7P6CZrMTZfOPdyq+OoCib3b8v7XIX/kb2QfNKBrVINihYAD1NSbuerrdlc5b8WiR8IgXqHDE8SHuRP5AVXs89EUvL9v6yOo9RZ0QLgYX5MOUDL0j20LN0LnfXiL09066AOvGcfTpOMXyAjyeo4Sp0xLQAeZunG/YzyX+t8osM/PVJ0eCAFXa/jqAmg5PsXrY6j1BnTAuBByuwOvtiaxbjAJIg5H0KjrY6kqnHzpefxX/tgfLfNh/xMq+ModUa0AHiQX3cdIqBwP22Lt2n3j4c7t1UIyW0nIcZB2arXrI6j1BnRAuBBlm7K5Ep/V5+yFgCPd83QQayw98G++k0oLbQ6jlI1pgXAQzgchuWbsxgfvA4izoXIc62OpE7jgvhmfN9iPAFledjXfWh1HKVqTAuAB1iQlE7fJ76k5MhBOhStY3uzi62OpNwgIlwydDQbHO0o/O5FcDisjqRUjWgBsNiCpHQe+HQjBwpKucyWhC8O/rqtHQuS0q2OptwwNCGKz4PGElKQikn50uo4StWIFgCLPbU8maIy5y0FLvdJJNM0Z3VZW55anmxxMuUOm03oOORGskw4h7963uo4StWIFgCLZeQWAdCEUi6xbeAL+/kYbMfblecbdX5bPvG9gub7f4SsLVbHUcptWgAsFh0eyGjbD/zY5I8ESQkjfX5htO0HosP1DqANRRNfH4IvmkKR8efgV/+0Oo5SbtMCYLFnOyczx+8NIuQIAJGSzz/83uD5hB0WJ1M1cfWAHiyWiwnd/gkcPWB1HKXcogXAYj22v0CQlJ7QFiilXLBTbzTWkIQE+JF/3hT8TBmHv3vl9Cso5QG0AFgsoLCa2wjkpdVvEHXWRg8dwneO8/Bd8yaUl1gdR6nT0gJgobyiMjJMi6pfDKtygjXlwVqGBJDS/iZCyg+Rn/iR1XGUOi0tABZatimT58rGYSq/4BcIlz1sRSR1lgaPnECyI5bi7/4F5qS/rFIexa0CICIjRCRZRFJEZGYVr4uIvOB6fYOI9Ha1dxKRdRV+8kXkHtdrs0QkvcJrV9TuR/N885PSCQ0JRQCCIwGBsDgY9QL0mGBxOnUmzmkZwupW19KycAdFO76xOo5Sp3TaSeFFxAd4CRgGpAGrRWSRMabigOeRQEfXTz/gZaCfMSYZ6FlhO+nA/ArrPWeMebo2PkhDk55bxKrUQzzR+ldoEgP3bAKbHpA1Bj2umMrBd17jyIrniD93iNVxlKqWO3ucvkCKMSbVGFMKfASMqbTMGOBd47QKCBeR1pWWuQzYaYzZc9apG4FF6zII5wjt8n6Gbtfozr8R6dEuim9CRtHmwHeUZqdYHUeparmz14kB9lV4nuZqq+kyE4HKt0yc5uoyektEqpz8VkSmikiiiCTm5OS4EdfzGWOYn5TGHZEbEEe5dvc0Qq2HTqPc2Niz5BmroyhVLXcKgFTRVvns1imXERF/YDTwcYXXXwba4+wiygSq/D/FGPOaMaaPMaZPZGSkG3E939bMI2zPKmCc788Q2RladbM6kqpl/c9L4LsmlxC7+1MchYetjqNUldwpAGlAXIXnsUBGDZcZCaw1xmQdazDGZBlj7MYYB/A6zq4mr7BgXTptbAdodXgtdP8dSFX1UzVkIoLPRXcRSDEpy/5tdRylquROAVgNdBSRdq5v8hOBRZWWWQTc6BoNdCGQZ4ypeIXTdVTq/ql0jmAcsKnG6Rsgu8OwcF0601ttcDZ0H29tIFVnBg26lLW2bjTb9DbYy62Oo9RJTlsAjDHlwDRgObAVmGeM2Swid4jIHa7FlgCpQArOb/N3HltfRIJwjiD6tNKmnxSRjSKyARgC/OlsP0xDsCr1IFn5JVxu/w5i+0KzeKsjqTri62PjcPcpRDpy2Pndf6yOo9RJTjsMFMAYswTnTr5i2ysVfjfAXdWsWwicdLmrMWZyjZI2EvOT0unVJIPQ/O0w0CtHwHqV/ldcz971c2DVSzDkRqvjKHUCHXtYj4pK7SzbtJ8/RiaB+EDCWKsjqToW1MSfnedMpn3JNvat/8bqOEqdQAtAPfpyaxZHS0q5qOgbaH8pNG0co5rUqfUcdSf5JoiDK3XGMOVZtADUowVJ6VzedA8BR9P15K8XadasOTsDunFe3tc4/hbG/lkdWL3oVatjKaUFoL4cLCjh2+053N58DfgGQucrrY6k6snqRa/SpTgJEbAJRJFDtzV/1SKgLKcFoJ58vjETHGX0yP8aOl8BTZpaHUnVk7i1TxEgZSe0BUopcWufsiiRUk5aAOrJ/KR0JrVIwbf4MHTXWz94k5am6luYtDQ6daSylhaAerD7wFGS9uYyuelqCGzmPAGsvEa2VH2yP1si6jmJUifSAlAPFqxLJ1iKaX/wG+fQT19/qyOperSv9wyKzIl/c4eBtB7TLEqklJMWgDpmjGFBUjp/iErGVl6kd/70QheMvp1N5z/OfiJxGOEAYQAE5mywOJnydloA6ti6fbnsPljI1X4/QWgsxF1odSRlgQtG307UrBRsj+QSMWsvX4ZPoGvmJ2QnLTn9ykrVES0AdWxBUjpRvgW0zvnJeedPnfhFAT1ufJJUE4PPZ9MxRblWx1FeSvdGdajM7mDxhkzuid6MGLte/KWOi2rRjC395hBefoB9H3rFfRCVB9ICUIe+35HDoaOljHB8Dy0TIEonflG/GTliFJ8GjafN3k85uuEzq+MoL6QFoA7NT8ogIfAw4QdcE78oVYGPTeg66e9sc8RhXzQdCg9ZHUl5GS0AdeRIcRkrNu/nT1GukR7dtACokyXEteSH7o8TWJbLwf/dY3Uc5WW0ANSR5ZuzKCm3M7Doa+fIn2ZtrY6kPNR1o69irt94WqQupGzTQqvjKC+iBaCOLEhKZ3B4DoG526GHnvxV1Qtu4ss54/6PjY54yhbeDUf1FhGqfmgBqANZ+cX8uPMAd7VYCzZfSBhndSTl4S7tGsuC+P/Dt/QIR+ffDcZYHUl5AS0AdWDRugwwDnrlfem870/wSTNiKnWSqb+7ipeYQHDKZ5hNn1gdR3kBtwqAiIwQkWQRSRGRmVW8LiLyguv1DSLSu8Jru12Tv68TkcQK7c1F5AsR2eF6bFY7H8l685PSmdgqA9+CDL3zp3Jbq9AAIi6/lyRHB8oW/RmOZFkdSTVypy0AIuIDvASMBBKA60QkodJiI4GOrp+pwMuVXh9ijOlpjOlToW0msNIY0xFY6Xre4CXvP8KWzHxubvor+AVBp5FWR1INyKT+7Xm9xf04yoooW/BH7QpSdcqdI4C+QIoxJtUYUwp8BIyptMwY4F3jtAoIF5HWp9nuGGCu6/e5QKOYIX3BunQCbHY6HlzpnPVLJ35RNeBjE6aNH8kz5RPw27kc1n9kdSTViLlTAGKAfRWep7na3F3GACtEZI2ITK2wTCtjTCaA67FlVW8uIlNFJFFEEnNyqp5Yw1M4HIaFSencEbMLW/FhvfWDOiMJ0aH49P8Dvzo6Uf75DMhLtzqSaqTcKQBSRVvl49JTLTPAGNMbZzfRXSJycQ3yYYx5zRjTxxjTJzKy6ok1PMWvuw+RkVfM7/xXQWBznfhFnbHpwzrzdMDdlJWV4lioXUGqbrhTANKAuArPY4EMd5cxxhx7zAbm4+xSAsg61k3kesyuaXhPsyApnUj/UmKyvoau48DHz+pIqoEK8vflD1cP44my67ClroS1c0+/klI15E4BWA10FJF2IuIPTAQWVVpmEXCjazTQhUCeMSZTRIJFJARARIKBy4FNFda5yfX7TUCDvgSyuMzO5xszuSd2B6ITv6haMKRzSw4nTOZnR1ccyx6Ew3usjqQamdMWAGNMOTANWA5sBeYZYzaLyB0icodrsSVAKpACvA7c6WpvBfwgIuuBX4HPjTHLXK/NAYaJyA5gmOt5g/X1tmyOFJcz0nwPYW0gtu/pV1LqNB4e1Y2/yZ2UlDswC+8Ch8PqSKoR8XVnIWPMEpw7+Yptr1T43QB3VbFeKnBeNds8CFxWk7CebH5SOuc2LabZ/h9hwHSd+EXVipahAUweOYhHFl/PnN1vQOKb0Pc2q2OpRkL3UrUgt7CUr5Oz+XP0JtfEL9r9o2rP9X3bkBw9jh/pifniYTi40+pIqpHQAlALPt+YSZndcHHxt9CyK7SqfJ2cUmfOZhNmX9ODv5ROodhug4V3gcNudSzVCGgBqAULktIZFFFAUPYavfOnqhOdo0K5atAFPFR8A+z9GVZVvtheqZrTAnCW9h0qZPXuw9wVsc7Z0O0aawOpRuvuyzqyOuxyfvTti1n5KORstzqSauC0AJylhevSAcP5+V9Am4sgvI3VkVQjFejvw2Nju3NPwc0USwB8MB6e6wqzwuG5brBhntURVQOjBeAsGGOYn5TO+Nhc/A7t0Hl/VZ0b3KklF56XwP9KLoTc3ZCXBhjI2weLp2sRUDWiBeAsbErPZ2fOUW4NTXRO/NJVJ35Rde//rurCZT5rTn6hrAhWPlr/gVSD5dZ1AKpq85PSaeIDnXKWQ4ehENTc6kjKC7QMCcBwsOoX89LqN4xq0PQI4AyV2x0sWp/B1LZZ2I5k6J0/Vb0qCoyqsr2wmnalqqIFoIYWJKUzYM5XdHhoKQcKShhc+jX4BevEL6pePVl2LYXG/4Q2Y2BeSX+LEqmGSAtADSxISueBTzeSnlsEgD9ltM9Zyb5WQ8A/2OJ0ypvMLejLzLIppDkicBgh09GcDNOc6+yLYduS029AKbQA1MhTy5MpKvvtCsxLbOsJl6M8n9XTwlTKG0WHB7LIMZCBpS9wTskH9C99kStK55Bii4f/3gAbPrY6omoAtADUQIbrm/8xY3x+4qAJYdGRcy1KpLzVjOGdCPTzOaEtj6Y80PRR7G0ugk9vg9VvWpRONRRaAGogOjzw+O9NKWSobQ2f2S+kZXiIhamUNxrbK4bZV3cnJjwQAWLCA7n+wjg2HjDcWjYDe4fL4fM/ww/PWR1VeTAdBloDfxrake8//TczfOcRIwcQgXxbKDOGd7I6mvJCY3vFMLbXidNz92nbnD/PW89N8dN5JyEY3y9nQXE+XPYwSFUztypvpgWgBoKSP2W23xsESenxtj/4fY6vzwhAbwGtrDeuVywA985bz41MYW7PEPx+eBZKjsDIJ3WeCnUC/dfgpk3peZy3/YUTdv4AvvZivfpSeZRxvWJ5ZsJ5rNqdy41Z11F24R9h9euw4A9gL7c6nvIgegTghpJyO/fOW89S0asvVcMwrlcsgvDneeu4kSuZe0kI/t8+AaUF8Lu3wLeJ1RGVB9AjADf888sdJGcdoSSoddULhMXWbyCl3DC2VwzPTujJL7sPceOOiykdNhu2fQb/mQClR62OpzyAWwVAREaISLKIpIjIzCpeFxF5wfX6BhHp7WqPE5GvRWSriGwWkbsrrDNLRNJFZJ3r54ra+1i1Z92+XF75difjz48lsNe1Jy/gF+g8waaUBzpWBH7ddYgbN/ei5KoXYdd38O5YKMq1Op6y2GkLgIj4AC8BI4EE4DoRqTzn4Uigo+tnKnBsuqJy4F5jTBfgQuCuSus+Z4zp6frxuMsXi8vs3DtvHa1CA3j4kjBIehdCoiE0BhAIi4NRL0APPQGsPNfYXjE8d62zCNy0tgMl496CjCR45yooyLE6nrKQO+cA+gIpxphUABH5CBgDbKmwzBjgXWOMAVaJSLiItDbGZAKZAMaYIyKyFYiptK7HevaL7ezMOcp7N51HyMIbobwUbl0BER2sjqZUjYzp6Rwu+qf/ruNGE8XcCR8S8L/J8PYIuHGhdmN6KXe6gGKAfRWep7naarSMiMQDvYBfKjRPc3UZvSUizap6cxGZKiKJIpKYk1N/31YSdx/i9e9TmdSvDYN2PgPpa2Dcy7rzVw3WmJ7OI4HVuw9x07dNKb7uEyjIhrdGwMGdVsdTFnCnAFR19YipyTIi0hT4BLjHGJPvan4ZaA/0xHmU8ExVb26Mec0Y08cY0ycyMtKNuGevqNTOfR+vJyY8kIdj10HiWzDgbugyql7eX6m6UrEI3PiljaLrF0JZobMIfP+sc2pJnWLSa7hTANKAuArPY4EMd5cRET+cO/8PjDGfHlvAGJNljLEbYxzA6zi7mjzCP5ZtY/fBQl681JeA5fdB/CC4VE/0qsbhWBFI3H2Im5aWUHjDZ87uzZWPOKeW1CkmvYY7BWA10FFE2omIPzARWFRpmUXAja7RQBcCecaYTBER4E1gqzHm2YoriEjFMZXjgE1n/Clq0arUg7zz025uv6AZPX/6IwQ2h9+9DT56yYRqPCoWgZsX5+HwCzx5IZ1istE77V7NGFMuItOA5YAP8JYxZrOI3OF6/RVgCXAFkAIUAre4Vh8ATAY2isg6V9uDrhE/T4pIT5xdRbuB22vtU52hoyXlzPjfeto1D+D+oucgLx1uWQpN66frSan6NKZnDCLCPR8lQZP9VS+kFzk2am59rXXtsJdUanulwu8GuKuK9X6g6vMDGGMm1yhpPZi9dCtph4v4/sJEfJJWwBVPQ9wFVsdSqs6MPi8agIxPWxArB05eIKSaix9Vo6BXArv8sOMA76/ay9+7ZRGb9Bz0uBYumGJ1LKXq3Ojzovk5/q4qp5gsK8yF7SssSqbqmhYA4EhxGff/bz39WxRwXdqj0KorXPW83j5XeY3ns3qeMMVkmiOCJ8qvY7ejJfxnPCx/yHmiWDUqemYTePyzrRzKz+fL6H8hBQ6Y8C74B1kdS6l6k5FbRDoDWVQ68IT2d4uGkzzoR/j5Rdj9g/NGci3aW5RS1TavPwL4Ojmb/ybu479x8wk6uBHGvaL/wJXXqTjbXUUl+HN/0WQOjXobDu+GVy/WoaGNiFcXgLzCMmZ+soHpzX7ivOyFMOg+6OyR96RTqk5VNcdwgJ+NwedGsCApg/7zA3m5y1zKW3Zzzje84E4oKbAoraotXt0F9Mhnm4k6uo17Al6Hc4bAkAetjqSUJY5NLfnU8mQycouIDg9kxvBOjO0Vw75DhTyzIpl//JzBm4H38Hq7r+i57nVk36/OLqHWPSxOr86UOEdwNgx9+vQxiYmJtbKtL7ZkMePdr/k2bBZhAb4w9VsIblEr21aqMdqUnsfspVv5MeUgo8J28qT8i4CyXOTyx6HvVB004cFEZI0xpk/ldq/sAjp8tJSHPlnHG01fJbT8IEyYqzt/pU6jW0wY7/++H3Nv7UtKUC8uyn2U1baesPR++GgSFB6yOqKqIa/sAnp40WYml35EH5+1zuGeMedbHUmpBkFEuOTcSAZ1iGDBunTuWRbJ8LIFPJj8Ebx0EX7j34T4AVbHVG7yuiOAJRszKdj4OX/0+RR63gDn32x1JKUaHJtNuLp3LF/NGELr4X/iBh4j7YjB8c5V5C97DBx2qyMqN3jVOYADBSXc8uzHfGjuJ7hVe+T3K5xTOiqlzkpuYSlvrNxAh9WzGGv7nr0hvcmKHUbc1rdoaXLIlkj29Z7BBaMtv+WXV6ruHIDXFABjDHe/9zN37PwDnQIO43P7t9C8XS0nVMq7pR0u5Lt5L3B1xpM0wX7CeeEi48+m8x/XImABry0Aqxe9Stzap2hpciiiCYFSgm3Sx3Du5XWUUimVPSuelhw+qX0/EUTN0tnH6lt1BaBRnwRevehVuq35K4FSCgLBlFBmfFizbRcXnGt1OqUarwhzuMr7ALcyB9j7WDcKQ9sTFJ1AVIfz8G/VGSI6gn9w/Qf1co26AMStfcq586/AT+zErX0K9DBUqTqTLZFEcfIc3gUSSJZvLC0OJRN96Dt8NzuOv1YaHINfVGckshNEnAuRnSCi04lDtDfMc05Sk5fmnMj+soehx4T6+EiNUqMuAC1NTpXfQlqaKu57rpSqNft6zyDs2NG3S5HxZ9v5j3DB6NspLC3nx5T9bNuynpxdGwnITaFDfjrnHk2lQ+oP+JuS3zYW1MJZEMQXx96fsZlyZ3vePsoX/tG5E9MicEYadQGo7ltItkQQZUEepbzFBaNvZzW4zr8dIFsi2Hf+b6OAgvx9uSQhlksSYoErOVBQws87D/JuygF+2J4N+Wl0kHR6BWbTz/8AHQoyaH5oFTZOPGfpay+mdNGf8Q9qAdG9IKh5/X/YBqxRnwQ+4RyAi45EUMqzGWPYe6iQH1MO8mPKAX7ceYDcwjJSm0zCdrq7TYS3dRaCmN7Ox9bnQUBYveT2ZDoK6Ni3EB2LrFSD4nAYtmTmE/5qb2JtJ3ffZjia838yjf6Be+lhS6VD+Q6al2Yef70svD0+sb2wHSsKUT2gSVPgxFGCjflahbMqACIyAvgnzknh3zDGzKn0urhevwLnpPA3G2PWnmpdEWkO/BeIxzkp/ARjzMnjxiqozZvBKaUallmP/437y/5NUIUj+kLjzyNyO4G9ryM9t4gM1w+FB+lu20V32UUPWyrdbalEi/NeRQ5sHAyMJ49g2hRuwV9+u2r5bHsI6qKg1MY2z7gAiIgPsB0YBqQBq4HrjDFbKixzBfBHnAWgH/BPY0y/U60rIk8Ch4wxc0RkJtDMGPOXU2XRAqCU91qQlM4P8//NPXxEtBwkw7TgeSYycNydx29nfUxRqZ2MvN8KQnpuMQU5aQQe3EjEka3EF2/jYlmHj5y8/7MbYY9EUyjBFNmCKfYNodSnKaV+Idj9QrD7h2KahGICwrAFhGILCsc3KIzS5C8ZtufZEwpUkfFnVbe/0WnY7/G1Cb4+Nnxsgp+POB9tNmyn6NeqrW7ssykA/YFZxpjhrucPABhjZldY5lXgG2PMh67nycBgnN/uq1z32DLGmEwRae1av9OpsmgBUMq7LUhKr3LOgpoyxmBmhVd5TsEY2NzsUvzK8vEvLyDAXkCA4yjBjgL8Kavxe9mNcIgQBBAMNszxRzAIYMPx2+vy2+s+xl7lXbb3E0nUrBS3M5zNhWAxwL4Kz9Nwfss/3TIxp1m3lTEmE8BVBFpWE3wqMBWgTZs2bsRVSjVWY3vFnNEOvzIRIauaUYJZEkm3e+ZXvWJZMZTkQ3E+jqJcSo4eprTgMGVHc2nx1Ywqd9Y2DHlthuPA2f1kN+Aw4nxuxPkcwWGcxaLi45Cc96uMUVtD2d0pAFUdn1Q+bKhuGXfWPSVjzGvAa+A8AqjJukopVZ3qrlXYd/6M6oeJ+wU4f5q2xAYEun4A9n/9ZLUFpcPv3zijjPtnLa/Toezu3A46DYir8DwWyHBzmVOtm+Xq+sH1mO1+bKWUOjsXjL6dTec/zn4icRhhP5FndQJ4X+8ZFBn/E9qKjD/7es8444x1sc2K3DkCWA10FJF2QDowEZhUaZlFwDQR+QhnF0+eq1sn5xTrLgJuAua4Hhee7YdRSqmauGD07cdvCxPl+jmbbZ3q4jdP2WZF7g4DvQJ4HudQzreMMX8XkTsAjDGvuIaBvgiMwDkM9BZjTGJ167raWwDzgDbAXmC8MeaUc8rpSWCllKo5r70QTCmlvJ1OCq+UUuoEWgCUUspLaQFQSikvpQVAKaW8VIM6CewaVrrnDFePADx9JhhPz+jp+cDzM3p6PtCMtcHT8rU1xkRWbmxQBeBsiEhiVWfBPYmnZ/T0fOD5GT09H2jG2uDp+Y7RLiCllPJSWgCUUspLeVMBeM3qAG7w9Iyeng88P6On5wPNWBs8PR/gRecAlFJKncibjgCUUkpVoAVAKaW8lFcUABEZISLJIpLimn/YY4hInIh8LSJbRWSziNxtdabqiIiPiCSJyGdWZ6lMRMJF5H8iss3137K/1ZkqE5E/uf7Gm0TkQxEJ8IBMb4lItohsqtDWXES+EJEdrsdmHpbvKdffeYOIzBeRcKvyVZexwmv3iYgRkQgrsp1Ooy8AronpXwJGAgnAdSKSYG2qE5QD9xpjugAXAnd5WL6K7ga2Wh2iGv8ElhljOgPn4WE5RSQGmA70McZ0w3l79InWpgLgHZy3ca9oJrDSGNMRWOl6bpV3ODnfF0A3Y0wPYDvwQH2HquQdTs6IiMQBw3De7t4jNfoCAPQFUowxqcaYUuAjYIzFmY4zxmQaY9a6fj+Cc8d19pOe1jIRiQWuBM5sbrs6JCKhwMXAmwDGmFJjTK61qarkCwSKiC8QxMkz69U7Y8x3QOV5OMYAc12/zwXG1muoCqrKZ4xZYYwpdz1dhXOmQctU898Q4Dngfmo4DW598oYCUN2E9R5HROKBXsAv1iap0vM4/zE7rA5ShXOAHOBtVxfVGyISbHWoiowx6cDTOL8NZuKcNW+Ftamq1coYkwnOLyhAS4vznMqtwFKrQ1QmIqOBdGPMequznIo3FICznpi+PohIU+AT4B5jTL7VeSoSkauAbGPMGquzVMMX6A28bIzpBRzF2m6Lk7j60ccA7YBoIFhEbrA2VcMmIg/h7EL9wOosFYlIEPAQ8LDVWU7HGwqAO5PaW0pE/HDu/D8wxnxqdZ4qDABGi8hunF1ol4rI+9ZGOkEakGaMOXbk9D+cBcGTDAV2GWNyjDFlwKfARRZnqk6WiLQGcD1mW5znJCJyE3AVcL3xvIuZ2uMs9Otd/8/EAmtF5GymHK4T3lAAjk9qLyL+OE+8LbI403Gu+ZTfBLYaY561Ok9VzP+3c4cqEURhFMf/pwpWk2HbVjEJxkUQg28gG6yaxReQfQCDLyCCWLSJYLWJuGix6QafwHoMc4Ori/XO7pwfDAyTDjP347uXO1z72Paq7R7N+7u33ZrZq+1P4ENSvzwaAK8VI83yDmxIWirffEDLNqp/uAGG5X4IXFfM8oekbeAI2LX9VTvPb7bHtlds90rNTID1Mk5bZeEbQNksOgBuaQru0vZL3VRTNoE9mln1U7l2aoeaQ4fAuaRnYA04qZxnSlmdXAGPYHVTVQAAAF9JREFUwJim9qofFyDpAngA+pImkvaBEbAl6Y3mL5ZRy/KdAsvAXamXs1r5/sk4F3IURERERy38CiAiImZLA4iI6Kg0gIiIjkoDiIjoqDSAiIiOSgOIiOioNICIiI76Bm/ZQDoxFxw7AAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/poisson_9_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "λ = 0.5\n", + "T = 10\n", + "\n", + "def poisson(k, r):\n", + " \"Poisson pmf with rate r.\"\n", + " return np.exp(-r) * (r**k) / factorial(k)\n", + "\n", + "@njit\n", + "def draw_Nt(max_iter=1e5):\n", + " J = 0\n", + " n = 0\n", + " while n < max_iter:\n", + " W = np.random.exponential(scale=1/λ)\n", + " J += W\n", + " if J > T:\n", + " return n\n", + " n += 1\n", + "\n", + "@njit\n", + "def draw_Nt_sample(num_draws):\n", + " draws = np.empty(num_draws)\n", + " for i in range(num_draws):\n", + " draws[i] = draw_Nt()\n", + " return draws\n", + "\n", + "\n", + "sample_size = 10_000\n", + "sample = draw_Nt_sample(sample_size)\n", + "max_val = sample.max()\n", + "vals = np.arange(0, max_val+1)\n", + "\n", + "fig, ax = plt.subplots()\n", + "\n", + "ax.plot(vals, [poisson(v, T * λ) for v in vals], \n", + " marker='o', label='poisson')\n", + "ax.plot(vals, [np.mean(sample==v) for v in vals], \n", + " marker='o', label='empirical')\n", + "\n", + "ax.legend(fontsize=12)\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Solution to Exercise 2\n", + "\n", + "Here is one solution. It shows that the approximation is good when $n$ is\n", + "large and $\\theta$ is small." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAANYCAYAAACRv3nwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXjc1X3o//eZfaTZtEuWZMsbtmyBwTE7ARJCakgc2iZhSSAtvS2lDST3Up4madLtNm3S50nS2yXLJaGkJLlQkqYECDVpAvwgAQJmly0bvGBr3zWafT2/P45GlmXJGq0zkj6v59Fjeb7LnBktH51zPudzlNYaIYQQothYCt0AIYQQYioSoIQQQhQlCVBCCCGKkgQoIYQQRUkClBBCiKIkAUoIIURRkgAlhBCiKEmAEmIZUEp9Vyn1xUK3Q4ilJAFKiAmUUk8rpeJKqfDYx6EJx8qVUv+plIoopY4rpT5WyLZOpJQ6Tyn1K6VUVCn1olJq7QLcc1av90zvnRBzIQFKiNPdobX2jH1smfD414EkUAN8HPimUmp7QVo4gVKqAXgc+HugAjgKfGEBbj2X1zvdeyfErEmAEsuSUup2pdRPlVJfV0oNKKW6lFJXL+LzlQIfBv5cax3WWv8SeAS45QzXfFYpdUQpFVJKHVBK/daEY+8ope5WSr2hlAoqpf5dKeWacPw8pdQrY9f+O+Ca8kmMrwLf1lo/orWOAQ8C5y/16xVioUmAEsvVOcDFmF+a1cD/BT4z8QSl1GNKqZFpPh47w72/NBb0fqWUunLssbOAjNb6rQnnvQ6cqUdxBHg34Af+Gvi+UqpuwvHrgd3A+rHX87tj7XYADwPfA8qBH2KCxWmUUj7gOuA7Ex62APEpzp3N+zGX1wtTv3dCzImt0A0QYo7OAb6stX4CQCl1ABMMxmmtPziH+34GOIAZ2roReFQpdS7gAYKTzg0C3ulupLX+4YT//rtS6nPABcBPxh77J61111j7HwXOHXv8IsAO/B9tqjn/SCl11zRPc9XYuW8opXKPOSc8x8T2zOb9mPXrZZr3Tmt9ZBbPK8Q46UGJ5eps4NEJ/2/B/HKcF631r7XWIa11Qmv9b8CvgGuBMOCbdLoPCE13L6XUJ5RSr+V6KWNtrJxwSs+Ez6OYoACwBujUp241cHyap2kCHtFaB3IfwFPA3jO+0JnN+vWe4b0TYk4kQIllRym1HtP7n5gldh7w2qTz/mtCRtnkj//K8+k0oIC3AJtSavOEYzuA/dO0cR3wbeAOoGIscLSO3Wsm3UC9mtAlAqbLynNiglvuedcDuzBDn5PbNJv3Y1avdxq5906IOZEAJZajc4A3tdbZCY+dh5kjGae1vmZCRtnkj2sm31QpFVBK/YZSyqWUsimlPg5cDjyhtY4APwb+t1KqVCl1KWbu53vTtLEU8wu6f+zet2J6UPl4HkgDnxprx29jhgan8hJwhVJqjVKqEfh/wOe11kOTT5zN+zHb13um9y7P1yzEaSRAieXoHCb0lpRSFUAtpocyH3bgi5igMgDcCfym1jrXU/tjwA30AQ8Af6S1nrJHobU+gMmuex7oxQxJ/iqfRmitk8BvY5ImhoEbMMFiKk9ihjrfAn4JfE9r/e18nicPZ3y9Yz2yPxv770zvnRCzpmRHXSGEEMVIelBCCCGKkgQoIYQQRUkClBBCiKIkAUoIIURRKspKEpWVlbqpqanQzRBCCLEEXn755QGtddXkx4syQDU1NbFv375CN0MIIcQSUEpNWSlFhviEEEIUJQlQQgghipIEKCGEEEVJApQQQoiiVJRJEkIIsViy2SwdHR1EIpFCN2XVsNvtVFdX4/NN3sHlzCRACSFWlYGBAZRSbNmyBYtFBpEWm9aaWCxGZ2cnwKyClAQoIYpMW3eQva29dI7EqA+42d1SQ3Odv9DNWjFGRkZoamqS4LRElFKUlJRQX19PV1fXrAKUfIWEKCJt3UHueeYYwViKOr+LYCzFPc8co6178u7rYq4ymQx2u73QzVh13G43qVRqVtdIgBKiiOxt7cXvtpHOZslqjd9tx++2s7e1t9BNW1FO3axYLIW5vOcyxCdEEekciZHOZOgOJvC6bDTX+fC6bHSOxArdNCGWnPSghCgidqvi2EAUr8tGKJ7m2ECEUDxFfcBd6KaJJdDU1MTPf/7z0x5/9tln2bJly5K3ZzbP+93vfpfLLrtsQZ9fApQQRaJ9KIpVKexWRX3ATX3ARftQlKP9EXa31BS6eaKA3v3ud3Po0KFV87w5MsQnRBEYjiR57I1uzqr18tFdDTx5sJ9IMkNDmRu/247dKn9LFppkVy49+a4XosDiqQw/ea0TpeC6HfXsaCzjf119Fl/56A7+8cbzOKchwN7WHnpH44Vu6qq1lNmVL730Etu2baOsrIxbb72VeDzO008/TUNDw/g5TU1NfOUrX+Gcc87B7/dzww03EI+f/P749re/zaZNmygvL+dDH/oQXV1d48eUUnzjG99g8+bNeL1e/vzP/5wjR45w8cUX4/P5uP7660kmkwCnPe+Xv/xlNm7ciNfrZdu2bfznf/7ngr/+iaQHJUQBZbKax97oZjSe5sPvasBfcmr6s81qYc+ONTzw4gkeea2LGy9oxOuSFOmF8vShPvpDiRnP+9n+HmLJDOH4yTTpWDLD1372Fu/fXnvGa6u8Tq7cUp13m37wgx/wxBNPUFpayp49e/jiF7/I+973vtPOe+ihh9i7dy8ul4tLL72U7373u9x+++08+eSTfO5zn+NnP/sZ27dv5+677+bGG2/kmWeeGb927969vPzyy7S3t7Nz506ee+45fvCDH1BRUcHFF1/MAw88wO/8zu+c9pwbN27k2Wefpba2lh/+8IfcfPPNHD58mLq6urxf32xID0qIAmjrDvIP/32IW+79NQ/ta2djZem0iRClThvXnVtPMpPl//5/R/jKE4e4+4ev8w///Zasj1oiw9EULvupvy5ddgvD0dmt68nHHXfcQWNjI+Xl5Xz+85/ngQcemPK8T33qU6xZs4by8nL27NnDa6+9BpgA93u/93vs3LkTp9PJl770JZ5//nneeeed8Ws/85nP4PP52L59Oy0tLbz//e9nw4YN+P1+rrnmGl599dUpn/OjH/0oa9aswWKxcMMNN7B582ZefPHFBX8PcqQHJcQSyw0XJdIZ0pksZSUOfn6wjw3VpdPOaVR5nZxV7eGr//0W1T4nOxr848NMt12+XuZC5ijfnk3HcIxgLIXffbL3Goyl2O6289FdjQvapsbGk/dbt27dKcNzE9XWnuy5lZSUjJ/X1dXFzp07x495PB4qKiro7Owkt1N5Tc3JpBu3233a/3t6eqZ8zvvvv5+vfe1r48EuHA4zMDAwuxc4C9KDEmKJ7W3txWlTDIaTVHicbK3z5rUYt7VrlA1VpSRSWQZCSVnEu4R2t9QQjKUIxlJktR7/fDGyK9vb28c/P3HiBGvWrJnV9WvWrOH48ZMb1EYiEQYHB6mvr59Xu44fP84f/MEf8C//8i8MDg4yMjJCS0sLWut53fdMJEAJscQ6R2Kk0lkA1leWAiqvxbidIzE2VpXitlsZjJhJbFnEuzSa6/zcdvl6/G473cE4frd90XquX//61+no6GBoaIi/+7u/44YbbpjV9R/72Me47777eO2110gkEvzZn/0ZF1544Xjvaa4ikQhKKaqqqgC47777aG1tndc9ZyJDfEIssfqAm5eODeFx2sbTx0Px9IyLcesDboKxFGWl5pdkOpslksjIIt4l0lznX5Kh1I997GO8//3vp6uri+uuu44vfOELs5rnueqqq/ibv/kbPvzhDzM8PMwll1zCgw8+OO92bdu2jT/5kz/h4osvxmKx8IlPfIJLL7103vc9E7WY3bO52rVrl963b1+hmyHEonj5+BCf/89W1leWsqnaQyieJhhLzfgXeW7uymaB44NR1gTcKKVkDmqW2traaG5uLnQzVqXp3nul1Mta612TH5chPiGWmMNqZefaAA3l7lkNF+WGmer8bmKpLKlMVoKTWNFkiE+IJXZ0IMymai+3Xto06wrPuWGm7fV+jvZH2FIzux1KhVhOpAclxBJKprOcGIyyvqp0Xls+bKgsJZ7KSIKEWNEkQAmxhE4MRUlnNRsrPfO6z9qKEqwWxdGByAK1TIjiIwFKiCV0tD+M026hvmx+mXdOm5XGcjdH+8OLug5FiEKSACXEEtFac2wgQlNFKVbL/Hd03VDpYSSaWpRyO0IUAwlQQiyRntE40WSGDVWlC3K/9WP3OdofXpD7CVFsJEAJsUSO9kewKEVTxcIEKJ/LTrXPydF+mYcSK5MEKCGWyNH+MPVlblx264Ldc0Olh65gjGgyvWD3FMvL9u3befrppwvdjEWRV4BSSu1WSh1SSh1WSn12iuNblVLPK6USSqm7Z3OtEKtBMJpiIJxcsOG9nI1VpWgNxySbb0VoamrC7Xbj8Xioqanh1ltvJRw+8xDu/v37ufLKK5emgUtsxgCllLICXweuAbYBNymltk06bQj4FPCVOVwrxIp3ZMD8ktlQubABqsrrxOuyyTDfUuhphae+BA//sfm3Z3EKpT766KOEw2FeeeUVXnrpJb74xS8uyvMsB/n0oC4ADmutj2qtk8CDwHUTT9Ba92mtXwImpxPNeK0Qq8HR/ggVHgeBEseC3lcpxfrKUrO+KpNd0HuLCXpa4bl/htgI+OrNv8/986IFKYD6+nquueYaWltbeeSRR9i+fTuBQIArr7yStra28fOampr4+c9/DsCLL77Irl278Pl81NTUcNdddwEQj8e5+eabqaioIBAIcP7559Pba7Zp6erq4kMf+hDl5eVs2rSJb3/72+P3/qu/+iuuv/56PvGJT+D1etm+fTtLWSc1n1JH9UD7hP93ABfmef+8r1VK3QbcBrB27do8by9E8YunMnQOx3jXurJFuf+GKg9vdARpH46Nbd8h8vb2zyGcx35aBx+DZBQSo5DbxDgZhaf+FrZ+8MzXempg8+lbts+kvb2dxx9/nHPOOYebbrqJhx9+mCuvvJJ/+Id/YM+ePRw4cACH49Q/eD796U/z6U9/mltuuYVwODy+Hca//du/EQwGaW9vx+l08tprr+F2m7V4N910E9u3b6erq4uDBw9y9dVXs2HDBq666ioAHnnkEX784x9z33338YUvfIE77riDF154YdavZy7y6UFNtWAj35WBeV+rtb5Ha71La70rt9+IECvB8cEoWa0XfP4pp7HMjcNm4diApJsvmugQ2Cctrra7zeML7Dd/8zcJBAJcdtllXHHFFWzbto0PfOADXH311djtdu6++25isRjPPffcadfa7XYOHz7MwMAAHo+Hiy66aPzxwcFBDh8+jNVq5V3vehc+n4/29nZ++ctf8vd///e4XC7OPfdcfv/3f5/vfe974/e87LLLuPbaa7Fardxyyy28/vrrC/6ap5NPD6oDmLincQMw9R7EC3utECvC0f4wJQ4rtT7XotzfZrWwtryEo/0R3rNFz6vG36qTb89m5IQZ1nMHTj4WG4G6HXDexxe0SQ8//DDve9/Jdv3RH/0R69atG/+/xWKhsbGRzs7O06699957+Yu/+Au2bt3K+vXr+cu//Es++MEPcsstt9De3s6NN97IyMgIN998M3/7t39LV1cX5eXleL3e8XusW7fulGG8yVvLx+Nx0uk0Ntvi1xrPpwf1ErBZKbVeKeUAbgQeyfP+87lWiGUvk9UcG4ywvrIUywJUj5jOhqpSQvE0/aHEoj3Hqta8B+IjJijprPk3PmIeX2STt3DXWtPe3j7lFu6bN2/mgQceoK+vj8985jN85CMfIRKJYLfb+cu//EsOHDjAc889x2OPPcb999/PmjVrGBoaIhQKjd/jxIkT894efqHMGKC01mngDuAJoA14SGu9Xyl1u1LqdgClVK1SqgO4C/iCUqpDKeWb7trFejFCFJuukRiJVJYNVfMrDjuT9ZWlKAVHJJtvcdS2wCV3mh7UaKf595I7zeOL7Prrr+enP/0pv/jFL0ilUnz1q1/F6XRyySWXnHbu97//ffr7+7FYLAQCprdntVp56qmnePPNN8lkMvh8Pux2O1arlcbGRi655BI+97nPEY/HeeONN7j33nv5+McXtlc4V3n10bTWjwOPT3rsWxM+78EM3+V1rRArXVt3kL2tvbx8fJhMJstVW6sX9flKHDbQ8O1nj/Afr3RQH3Czu6VGNjNcSLUtSxKQJtuyZQvf//73ufPOO+ns7OTcc8/l0UcfPS1BAmDv3r3cddddRKNR1q1bx4MPPojL5aKnp4fbb7+djo4OPB4PN9xwAzfffDMADzzwALfffjtr1qyhrKyMv/7rv+bqq69e6pc5JdnyXYgFltua3e+2caQvjFKKCo9zUXe/besO8uXHDzISS3HRhnISaZ3XNvKrkWz5Xjiy5bsQBba3tRe/247DaiGZ0awJuPG77extzSOdeR7PWRdw4bJbCcbS+N32RX9OIRabBCghFljnSAyvy8Zo3NTHC7jteF22Rd39tnMkRpXXicNmIRQ36+UX+zmFWGwSoIRYYPUBN6F4mnAijc2qcNothOJp6gPz26Rw5ufM4HHaCCdMYFzs5xRisUmAEmKB7W6pIRhLMRBKUOowQ27BWIrdLTWL/pwKiCUzDIUTi/6cQiw2CVBCLLDmOj+/e8k6NJBIa/xu+6InKzTX+bnt8vXU+FyEExksFiUJEmdQjMlhK102O/takYu/FFiIVai81MlFGyr4rfPqaVqi+njNdX4+e62Hbz59hAvXV0hwmobL5WJwcJCKigqpurEEtNakUil6e3spLZ3dz4IEKCEWQc9oHIBa/+KUN5qO02alotRB79jzi9M1NDTQ0dFBf39/oZuyathsNvx+P5WVlbO7bpHaI8Sq1hOMEyixL+juufmq8bk4OhBBa6nLNxW73c769esL3QyRB5mDEmIR9I7GqVvi3lNOrd9FLJlhNCbbwIvlTQKUEAssFE8RiqepWaTq5TPJDSt2j8oaKLG8SYASYoH1Fmj+Kaey1IndqugJyjyUWN4kQAmxwHqCCawWRZXHWZDnt1gU1V6XJEqIZU8ClBALrGc0TqXHic1auB+vGr+LvtEEmays9xHLlwQoIRZQNqsLmiCRU+d3kc5qBsKygaFYviRACbGAhqJJkulswRIkcnLPL/NQYjmTACXEAsoFhEIlSOT4XDZKHNbxBcNCLEcSoIRYQL2jcZx2C2Ul9oK2QylFrV8SJcTyJgFKiAXUMxqn1ucqigoONT4XQ5Ek8VSm0E0RYk4kQAmxQFKZLAOhJLUFnn/KqfO70Br6RiVRQixPEqCEWCB9oQRZrakp8PxTzniihAzziWVKApQQC2Q8QaJIelAuu5WyErsEKLFsSYASYoH0jsbxue2UOotnk4Bav4ueYEw26BPLkgQoIRZIdzBeNL2nnBqfi0giQyghlc3F8iMBSogFEE2mGY2lqPUXpv7edOr8bgB6ZcGuWIYkQAmxAHLzT4WuIDFZpceB1aJkHkosSxKghFgAPaNxLMpUES8mNquFKq9TSh6JZUkClBALoHc0ToXHgcNWfD9StT6XSYGXyuZimSm+nyYhlhmtNT3BRNElSOTU+Fwk01kGI8lCN0WIWZEAJcQ8jURTxFOZgheInU5u6w+pyyeWm7wClFJqt1LqkFLqsFLqs1McV0qpfxo7/oZSaueEY/9LKbVfKdWqlHpAKVWcP8VCzFEuAaHYEiRyAiV2nHaLzEOJZWfGFYVKKSvwdeBqoAN4SSn1iNb6wITTrgE2j31cCHwTuFApVQ98CtimtY4ppR4CbgS+u6CvQogC6hmN47BZqCh1zOKiVmh7FILt4G+E5j1Q2zLzsTlQSlHrc0kmn1h28lnyfgFwWGt9FEAp9SBwHTAxQF0H3K/NcvUXlFIBpVTdhOdwK6VSQAnQtWCtF6KA2rqD7G3t5ZeHBygvsXOod5TmOv/MF/a0wnP/DK4A+OohNmL+f8md5vh0x+YRpBKpDI++3sUrx4dpLC9hd0tNfm0VooDyGeKrB9on/L9j7LEZz9FadwJfAU4A3UBQa/2zqZ5EKXWbUmqfUmpff39/vu0XoiDauoPc88wxhqNJbBZQCu555hht3cE8Ln7UXDD4NrS/CP0HYegYPPF58zF0DAYOQbgH3AETrNoenVdbnzzYTzyVweu2EYyl8m+rEAWUT4CaamObyfmqU56jlCrD9K7WA2uAUqXUzVM9idb6Hq31Lq31rqqqqjyaJUTh7G3txe+2Y7daAEWNz4XfbWdva+/MF3e/DsMnQFmgtNJ8+BvMT5HCfG53w+AR8+H0muG+ebS1xufEZbcSS2bwu+35t1WIAspniK8DaJzw/wZOH6ab7pz3Ace01v0ASqkfA5cA359rg4UoBp0jMer8LvpDZq+lUqcNu9VC50hs+ou0hneehcQo2JxQtwMsYz+CsRGo3Hzyc5cfho/BaCdEB6F+5/T3zbOtDpuF8FhNPq/Ldua2ClEE8ulBvQRsVkqtV0o5MEkOj0w65xHgE2PZfBdhhvK6MUN7FymlSpTZYvQqoG0B2y9EQdQH3ITiaSKJNDaLwmmzEIqnqQ+4p74gm4GDj8E7v4KtHwRPFSTCoLMmIMVHTDJE8x7zeTwIZU3gqYVI39h5w/Nqa6nTRmQsQJ2xrUIUiRl7UFrrtFLqDuAJwAr8q9Z6v1Lq9rHj3wIeB64FDgNR4NaxY79WSv0IeAVIA68C9yzGCxFiKe1uqeGeZ44xEE7icVoJxtIEYyluOL/BnDAxE89bZ3pM2QysfzesuxR695+aqbfzlpNJEJfcefJY5WbYcSP07YeX/w1qtkPHvlll+OXaitbEkhmGI0lCifTJtgpRpFQx7hOza9cuvW/fvkI3Q4gz2t8Z5G8eO4DdZmHn2rKTmXETs/RsDuh8xfSK3n03tPzW3J4sOgS/+kc4/Auo2gLlGyA+au6bR4ZfW3eQB15sZ987w1y0oZyP7mqQLD5RNJRSL2utd01+vHh2VhNiman2ubhwQwXXnF3L1lrfyQNtj5rgZHeZhAiLBWrPNtl6c1VSDhYrlFRCqBscHvDWnny+GQJUc52fu3+jhHufPcZ7tlZLcBLLgpQ6EmKO+kJm4WuVZ9IeUMF2cPlg6KgZ1qvdYYbj5pGJB0CoBxreZYLf8FFIJ8zz5Hlfr9OG22GlTxbsimVCApQQc9QfSmC3KspKJlWQ8DdC8ITJvgusBUepGY7zN059o3z5G01iReUmkxE4eNgkU+R5X6UUVR4n/eHE/NohxBKRACXEHPWFElR6nFgsk5YBbrkW+t8ClBmGm5ilNx+5DL9UwgS+0S4YOT6r+1b7nAyGk2Rk6w2xDEiAEmIOtNb0hxJU+6bY4j0ZhrpzoWabmS9yB+Zdqggw119yp7mf1uCpNMkS5RvyvkW110UmqxmMSC9KFD9JkhBiDoKxFMl0lirPpArmsRE48ZxJJ2/57YV/4tqWk4Eu3Af77oMjv8i7F1XlNQG1bzRRdLv/CjGZ9KCEmINcBYnTelBHnjT/brpq8RvhqYa1F5m09qGjeV1SVmLHYbPIPJRYFiRACTEHfaEEFqVO3WJj6Cj0H4K1l5hSRUth3aVQUgGH9kJ65h1zlVJUehz0j0qAEsVPApQQc9AfSlDucWCzjv0IZTPw9s/BXQaNFy5dQ6w22HKNyeY79kxel1R7XfSHExTjIn0hJpI5KCHmoC8UZ2156ckHOl4yaeXnXG+CxlIKNJpisgd/auajcqnn05RBqvI6SbZnGYmmKJvNJotCLDEJUELMUiSRJpLIsDZ1FJ66zwzthXpg43uhYmNhGlVaBV2vmAoTay8640aH1WOJEv3hhAQoUdRkiE+IWeoLJagIv826Q/eaQJBOQCoGPa+bhIVCeOsJqNoKaBMsz7DRYXmpA4tS9Mk8lChyEqCEmKX+UILNQ0/h9JabnXGjA6a6Q2nNvHa+nZdgu1m86y6D0Q7QmWnLINmsFio8DvrDUvJIFDcJUELMUl8oTkW6F5s7YDYUtNjNLrizqIu34PyNJ8spZVKmF3WG8kpVXid9o5IoIYqbBCghZqk/lED7G81GgtEhU85IWRem3t5c5cogaQ1OHwwcNhscTrOAt9rrJJrMEElmlrihQuRPApQQsxBPZRiJpkhs+gAMHYNUHDw1C1dvb64mlkGy2swWH2e9f9rySicrSsgwnyheksUnxCwMjFVg8DZug76tEO41c1CTd8UthFwZJK3h5e9CpB+yWROsJskFqP5Qgg1VniVuqBD5kQAlxCz0jZU4qom+bRISLv2UmX8qJkrBukug9cfQd2DKoOm0WQmU2MdfjxDFSIb4hJiF/lCCUoeFkr7XwFcHvvpCN2lqlWdBaSWceN70qKZQ7XWN1xQUohhJgBJiFvpCCZosvaZqRP0u01spRkrB2oshMgADb095SpXXSTCWIp6SRAlRnCRACZGndCbLUDhJU/yg2SW3urnQTTqz6m0maeL4r6bsRVVPmIcSohhJgBIiT4ORJI7kMBWJdlP7zmItdJPOzGIxZY9CPTB87LTD45l8EqBEkZIAJUSe+kMJakP7KXE5zY65y0HN2eD0wvHnTjtU6rThcdqkByWKlmTxCZGngZEgdbG3ca25CJzLJDXbajPbf7z2/0ydwMToKZXOq7xO+kOyFkoUJ+lBCZGnVMdreO1ZVOMFhW7K7Fjt0Pmy2UzRV3+y0nlPK9VeJ0ORFKlMttCtFOI0EqCEyEM2k8HZ9xrWskZT2mg5OfRfUNYEmQQkI6dUOq/yOslqzWB45t14hVhqEqCEOJOeVnjqS6QeuJn1/U9R4qsodItmL9hu9qmy2ExxWxgvbFvtdQGSySeKkwQoIabT02qGwmIjJFMplM5S3fnzwu35NFf+RkhGTc3AyIDpSY0VtvW5bTjtFvpkHkoUIQlQQkyn7VEzFGa1k4kGGXU34vCUF27Pp7nKVTq3l4DOwsCR8cK2SimqPE7pQYmiJAFKiOkE281QWLiHZFaT9dRgcfsLt+fTXOUqnXtrAQ3JEFz0yfEafdU+FwPhBNms7A0likteaeZKqd3APwJW4Dta6y9POsFTQdEAACAASURBVK7Gjl8LRIHf1Vq/MnYsAHwHaAE08Hta6+cX7BUIsVj8jQwO9BDueIf2qINRnaKupJ+KigLt+TQfuUrnZ38E3vwR2Bzjh8KxFM+8PcChnhAbqjzsbqmhuc5fwMYKYczYg1JKWYGvA9cA24CblFLbJp12DbB57OM24JsTjv0jsFdrvRXYAbQtQLuFWHRHq95D5/G3UYkwIWsZ7kyIIyc6OVr1nkI3be7KN4LLb9LOgbbuID9t7SaRyuBx2QjGUtzzzDHauoMFbqgQ+Q3xXQAc1lof1VongQeB6yadcx1wvzZeAAJKqTqllA+4HLgXQGud1FqPLGD7hVg0P+kup9t3LglbKS4VR5UE2LfmZn7SXV7ops2dxQJrzoORExDuZ29rL1UeJyUOK7FkBr/bjt9tZ29rb6FbKkReQ3z1wMRB9w7gwjzOqQfSQD9wn1JqB/Ay8GmtdWTykyilbsP0vli7dm2+7Rdi0fQOjXCBNc1T5dfzi9Q5nN9UjlLQPRIrdNPmp+4ceOdZ6HqFzpFa6vwuShy28e3fvS4bncv9NYoVIZ8e1FT7CUyeTZ3uHBuwE/im1vo8IAJ8dqon0Vrfo7XepbXeVVVVlUezhFhc59g7SaVSvK2aKHFYsVoUoXia+oC70E2bn1wl9p43WeuzEoqnKXVaiSTSgF4Zr1GsCPkEqA5g4qxwA9CV5zkdQIfW+tdjj/8IE7CEKHrvCfQyoH0ci5VS4rASjKUIxlLsbqkpdNPmb81OyKTYUztIMJZCY7YT6RtNrJzXKJa9fALUS8BmpdR6pZQDuBF4ZNI5jwCfUMZFQFBr3a217gHalVJbxs67CjiwUI0XYtFEh6hTw2x/16VYrFZiqSx+t53bLl+/MjLcfGvAW8v6xFvc9u4man0uwokMFotaOa9RLHszzkFprdNKqTuAJzBp5v+qtd6vlLp97Pi3gMcxKeaHMWnmt064xZ3AD8aC29FJx4QoTj1vglI41pzNxcEQN17QSJ1/BQ17KWX2tDr4OM3uET53TTPfePowZzcEJDiJopHXOiit9eOYIDTxsW9N+FwDn5zm2teAXfNooxBLS2vobYWy9fTE7ViUotLjLHSrFl71NjjyJHS+gqWsiSqvk95RKXkkiodUkhBispETplZdbQu9o3EqPA7s1hX4o2K1Q+05MPA2xEep9rnoD0lFCVE8ZMNCISbreRNsDnTFZnrb2tlUvUw2J5yL+p3Q9hj8159ybiyDK+ZndN3HCGyQXCZReCvwz0Ih5iGdhP6DUNXMaFIRT2Wo8a3A4b2cYCf0HYChYzjL6nGmR+H5f1l+FdvFiiQBSoiJBt6CTMoM741tQVHjcxW4UYuo7VEobwKLFXc6SNrhJ6RKl1/FdrEiSYASYqLeVlOrzt9I72gcq0VRUeqY+brlKthu9ouyuVDhHkodVoJZ9/Kr2C5WJAlQQuTER2H4HVP1Wyl6RxNUepzYVmKCRI6/EeIhsxVHPIjXmiIdHSHrayh0y4SQACVEblt3/uN/wLFnQFnRWtMXiq/s+Sc4uZmhzQUaAvETOFIhRtbtLnTLhJAAJVa58W3dhyGbBYsNXrmf0XdeI5HKruz5Jzi5mWFpFVgULh3n5bqb6HJtLHTLhJA0c7HK5bZ1t1ghHYOKTWBzk2z9Cbg+RvVK70HByc0Mh45hf/0BHBE7faE4IBUlRGFJD0qsbrlt3SP9oCxQWgkuH+nhE9gsiorSVRCgcsqaUO5yNqQO0zuaKHRrhJAAJVY5fyPEgiZAucvAYof4KIPWaqq8TqyWqXaSWaGUgjXnUp3tJTzYQ0YqSogCkwAlVrfmPRDqNBl8JZUQGyEbH+FN3xUrf/5pKrVnU+pyUhE6wGBYelGisCRAidWttgXWXQrOUjMH5Q4weu4f0uPauDrmnyZzlOKua6Yq8ja9I+FCt0ascpIkIVa3TBrScbjgD6H5gwB0dY0CPauzBwWUNL0L1+u/JtLRCmuvLHRzxComPSixug0dgXQCaraNP9QbimO3KspLVnAFiTNQZU3YvZXortcK3RSxykmAEqtb735wlECgafyhvtE41V4XltWUIDGRUljqz0UFO0iP9ha6NWIVkwAlVq90AgaPQFUzWMyPQjar6Q8lVuf80wSla88lqyyMHt1X6KaIVUwClFi9Bt6CbPqU4b3BSJJURq/a+aecqopyhtzrSXS8bqq7C1EAEqDE6tV7wFQu99WffGh0FWyxkQev00awrIVoNGL2xxKiACSLT6xOibCpXL72QrNAdUxfKI7DZqGsxF64thUBpRSl1euJ9kThyb+FknKzqLl5j0nNF2IJSA9KrE79h0BnoXr7KQ/3jiao9jpRapUmSEzQlHkHz0gb2XCfqVcYGzGFdWW3XbFEJECJ1alvP3iqzMeYTFYzEEqs+uG9nLV9vyDorCdlcUKkD9wBE6hkt12xRCRAidUnNgzBztN6T4PhBOmsJEjkeOI9xOx+orYyCPdCNmMK68puu2KJSIASq09fm/m3uvmUh3MVvFf8JoV5cpSvxaOjBG2VJjhF+kzNQn9joZsmVgkJUGL16d0P/gYzZDXx4dE4TrsFv3t1J0iMa96D3xIhlkiCvcRU3YiPmEQJIZaABCixuoT7IDJwytqnnN5QnBqvSxIkcmpbGN5xOyN4yCgLaA3nXC9ZfGLJSJq5WF1695uNCau2jj/U1h3k8Td7ePJgH2fVeNm2xktznewmC9BfspF70h+mRitu4Cc09fbTsKnQrRKrhfSgxOrQ0wpP/R384m+g5w0YOgaY4HTPM8foHY1T6rCiteaeZ47R1h0scIMLr607yCOvd5NIZXA6Xbxj28gbr/6agye6C900sUpIgBIrX0+rWb8z0gE2J9hc4+t59rb24nfbsSiFUopavwu/287eVimSure1l/JSBz63nWgyQ7jibEps8PpLzxa6aWKVyCtAKaV2K6UOKaUOK6U+O8VxpZT6p7Hjbyildk46blVKvaqUemyhGi5E3toeNet3MgmwWKGsaXw9T+dIDK/LRiiewm5VOG0WvC4bnSOxQre64HLvjddpIxRPE7MHiHkacPW/DtlsoZsnVoEZA5RSygp8HbgG2AbcpJSaPMN8DbB57OM24JuTjn8aaJt3a4WYi2A7OD0Q6YeSCrDYxtfz1AfchOIpRmNpfG47oAjF09QH3IVudcGZ9yaN120nmc6SSGU5at9CtT1hMvqEWGT59KAuAA5rrY9qrZPAg8B1k865DrhfGy8AAaVUHYBSqgH4APCdBWy3EPnzN5oglU2Dp9o8NraeZ3dLDQPhBKPxFB6njWAsRTCWYndLTWHbXAR2t9QQjKUAjdaarmCMI7qODfU10PlKoZsnVoF8AlQ9MHHpeMfYY/me83+APwVkTEAURvMeGDlhFps6/aam3Nh6nuY6P7+xrRan3UosmcHvtnPb5esliw9orvNz2+XrqfW5iKWyZDX8wRWbqNlyEQwdhehQoZsoVrh80synWhSi8zlHKfVBoE9r/bJS6sozPolSt2GGB1m7dm0ezRIiT+UbTFmjRBBCXaZHtfOW8fU8dpuV926t5g8v3yBroCZprvPTXOdnU7WX/lDCBO7EuXD8Oeh6FTZdVegmihUsnwDVAUysbdIAdOV5zkeADymlrgVcgE8p9X2t9c2Tn0RrfQ9wD8CuXbsmB0Ah5q6/DUor4Yo/Be/pQ3edIzHqA24JTmdQX+bmcF+Y0XgKn8sLlZtNuv76y8EqlTfE4shniO8lYLNSar1SygHcCDwy6ZxHgE+MZfNdBAS11t1a689prRu01k1j1z05VXASYlH1tJoAlZt/miAYSzEaS9FQJkkRZ9IwljTSOTyW3Vi/E1Lxk3UNhVgEM/agtNZppdQdwBOAFfhXrfV+pdTtY8e/BTwOXAscBqLArYvXZCFmIToEo12w8T2nbEyYk/uFWy8B6owqPU6cdgudwzGa63wQWAfpBDz1RSiplM0MxaLIq9SR1vpxTBCa+Ni3JnyugU/OcI+ngadn3UIh5qO31QSmmu1THu4cieG0W6jySAXzM7FYFPUBNx3DUfNA737z3kaHoGzDyc0ML7lTgpRYMFJJQqxcWpvhvbImcHqnPKVjOCrzT3lqKHMzHE0RTqTN4ueydeAoMXtFyWaGYhFIgBIrV7Ad4kGomfov+nAizUhU5p/yVR8oAcaGRYPt4C4HT41ZAJ1JymaGYsFJgBIrV0+ryTCrPGvKw7nhqoaykqVs1bJV7XXisFnoHImaOaf4KPjqTU91tEs2MxQLTgKUWJkyKZNeXrUFbI4pT+kcjuGwyfxTviwWxZqAy/SgmveYxc7pBLjLzMLd2JBsZigWlAQosTINHoZ0ctrhPTi5/slikfmnfNUHShgIJ4mVN5uECHfAFOC12mHT+yRBQiwo2bBQrEw9rSYxIrBuysPRZJrBcNKkTIu85ebrOkeibKptORmQXrkfooOmyrlF/u4VC0O+k8TKk4yYIaea7dP+shxf/yRVy2elxufCblW0D0/ajqTxQpOQMnCoMA0TK5IEKLHy9B4AnT3j8F7HcAy7VVHjcy1hw5Y/q0VR63efrCiRU7EZSsrhxAsmaUKIBSABSqwcPa3w1JfgiT8zhUzD0++K2zESo87vxirzT7PWUOZmIJwgnsqcfNBigYbzIdRjKscLsQAkQImVIbet+2iX2ZDQ7h7f1n2yeCrDQCgh65/mqD7gRmtO33W49mzzvre/WJiGiRVHApRYGSZu664sZouNaSobdEj9vXmp87uwWdTpw3xWO9S/y2RQRgYK0zixokiAEitDblv3cK+ZC7E6pq1s0DkSw2ZR1Mr805zYrBZq/K7xQH+K+neZHqz0osQCkAAlVgZ/I4wcNwt0vXXmsWkqG3QMR6n1u7BZ5dt/rhrK3PSF4iTSmVMPOEqg7hxTSDYRKkzjxIohP6FiZWjeA8PHzV7PLt8p27pPFE9l6A8lZHhvnhoCJWgNXSPxKQ6eb7IoO19e+oaJFUUW6oqVwVNt0sozKZMoMWlb95zuYBytzS9YMXe1fhcWZeah1leWnnqwpBxsLnj2a/Dag6bquewVJeZAApRYGbpeBd8auPiTZphpGh3DUawWRV1A5p/mw2GzUOt3msKxk/W0mt5TIgx+q+wVJeZMhvjE8pdOQM+bUN18xuAEpoJEjc+JXeaf5q0+UEJPMEEynT31QNuj4F1jerWhLjPkKntFiTmQHpRY/npbzdDemvOmPaWtO8hP3+jhyYN9bK31ck6Dn+Y6/xI2cuVJpNL86sgAB3tGOavGy+6WGvOeBtvNNhyBteZrE+oBb63sFSVmTf6MFMub1mZ4z1tjhvim0NYd5J5njtEdjOJxWgHNPc8co607uLRtXUHauoM88kY3iVQGl81CMJY6+Z7m9opyB8Dlh+AJiA3LXlFi1iRAieUt2AHhflizE6bZtn1vay9+t52sVliUYk3Ajd9tZ2/r9KWQxJntbe2lrMRBhcdJKJHG77affE9ze0XFgqYXFQ+Z4r2yV5SYJQlQYnnregVsTqjeNu0pnSMxvC4rQ5EEXrcdq8WC12U7vVSPyJt5T22UldgJxdMkM9mT72lty8m9ohIhM9xXsQkqNha62WKZkTkosXwlI9B/yMw9TbNrLpjacT2jMeKpLHV+s/4pFE/LVhvzUB9wE4ylqCh10DEcYyiSxG23nnxPJ+4VFe6Dff9qKp1vfE/hGi2WHelBieWr+w3IZs6YHAGwu6WGruEY8VSGQImdYCxFMJZid0vNEjV05dndUkMwliKZ0bjsFjqHotO/p55qk2HZuc+knguRJwlQYnnKZk1yRNk6KK0846lba3201Pup9rkYCCfxu+3cdvl6yeKbh+Y6P7ddvh6/2w4oUlnNLRetnf49bXq3+Zodf25J2ymWNxniE8vT0FGzg2seQ0YD4SQOm5U73rOJHY2BJWjc6tBcZ1L1B8IJvvf8caxn2uq9pBzqdkD3a9B4PrjLlq6hYtmSHpRYXnKbEj56J5z4NaRTM17ydl8IpWBTtWcJGrj6VHqcVHgcvNU7Q3HYdZcACt755ZK0Syx/EqDE8pHblDDcA1qZ7TVe+PqUmxLmaK15uzdMfcBNqVMGDBbLpmoPnSMxIon09Ce5fFC/E3r3m6UBQsxAApRYPnKbEqZiZs1TxaYZS+gMRpIMRZKcVeNdwoauPmfVeNEaDvfNkASx9mKIDsJP74KH/9j0hs/wB4ZY3SRAieUj2A4ON4S6obTCrH+aZlPCnLd6ZXhvKVSUOigvdfD2TAFq6CgMHIZgJzh9JwvJSpASU5AAJZYPfyMMvG1Sy/1rzWPTbEqYc7hPhveWglKKzTUeOoajZx7ma3sUKjaA0wsjJ8Dtl0KyYlp5BSil1G6l1CGl1GGl1GenOK6UUv80dvwNpdTOsccblVJPKaXalFL7lVKfXugXIFaRTVebX2p2t/mYZlPCnIFwgsFwks0yvLckNlfnMcwXbAd3+VgJpBGIDMzYCxar14wBSillBb4OXANsA25SSk2uK3MNsHns4zbgm2OPp4E/0Vo3AxcBn5ziWiHyk4pA/S6oaobRTlNK5wx7DL3dG0Yp2CzDe0ui0pPHMF+ukKy3DhweGDoCsSEpJCumlM+4xwXAYa31UQCl1IPAdcCBCedcB9yvtdbAC0qpgFKqTmvdDXQDaK1DSqk2oH7StULMLBmFjpdgw+Ww/bfyuuTtvhBrZHhvySil2Fzt4cV3hogm05Q4pnjfm/eYOScwtflOvGjKVe2+dWkbK5aFfIb46oGJ/e+OscdmdY5Sqgk4D/j1VE+ilLpNKbVPKbWvv19SUMUkHS9CNg3rLsvr9MGx4T3J3ltam2fK5ptYSDYehMqNULHZLBkQYpJ8/rScag8DPZtzlFIe4D+A/6m1Hp3qSbTW9wD3AOzatWvy/cVqloxCxz6o2gKeqrwueWtseE+y95ZWpcdBWYmdt3rDnNMwTdWOiYVkMyl46TtwaC+c/z/Aal+6xoqil08PqgOYOEDcAHTle45Syo4JTj/QWv947k0Vq9Yse08Ah8eG9zwyvLeklFKcVeOlYzhKNHmGbL4cqx3O2m02NDz+q8VvoFhW8glQLwGblVLrlVIO4EbgkUnnPAJ8Yiyb7yIgqLXuVkop4F6gTWv9tQVtuVgdxntPW/PuPQ2GEwyEk5IcUSCbajz5LdrNKV8PtWeb0lXhvsVtnFhWZgxQWus0cAfwBNAGPKS13q+Uul0pdfvYaY8DR4HDwLeBPx57/FLgFuC9SqnXxj6uXegXIVaw8d7TpXlf8nbfWPaezD8VRJXHSVmJnbd7Z7G1xsb3moXXh/7LVD0XgjyrmWutH8cEoYmPfWvC5xr45BTX/ZKp56eEmNkse09t3UH2tvbyzFv9VJQ6aB+KyJYaBaCUwmmz8NC+dn55uJ+15aXsbqk589fCUQKb3gcv3Wu+5jpjUs+b90y7jECsfFJJQhSfXMXyf/84HHkKSs683xOY4HTPM8foHY1jtyqcdgv3PHOMtu7gEjRYTNTWHeTZwwPEUxkcVgvBWCq/r4XW0HcA+tqgpELKIAkJUKLI5CqWRwbMVhp2N7z6vRl/Se1t7cXvtpPKZFFKsba8BL/bzt7W3iVquMjZ29pLtddJoMTOUDSF323P72tx8DGo3gp2FwwdkzJIQgKUKDK5iuXJEKDNVuF5/JLqHIlR4rDSOxrH77bjsFnxumx0jsSWpt1iXOdIDK/LTpXHyWgsRSSRzu9rEWyH0mooazLVJYKdUgZplZMAJYpLsB0sVhjtgtIqsJfk9UuqPuDmncEIqYymocwNQCiepj7gXopWiwnqA25C8TQ1fhc2i6JjOJrf1yJXBsm3BkorYfgd83WXMkirlgQoUVx8DdD7JiirST+GGSuWA1y1tZoTg1HsVgulThvBWIpgLMXulpolaLSYaHdLDcFYikgiQ63PSXcwTt9ofOavRfMeU0A2FoTyTeax3v2w4YrFb7QoShKgRHEpa4LoEHiqwWKbsWJ5Tiqb5ZwGP2fVeOgOmmG+2y5fL1l8BdBc5+e2y9fjd9vJovA4bbTU+2b+WkwsgxTugfp3mQ0O+w5AOrk0jRdFRZbZi+IRHTJVyls+AukYBDtMz2nnLWdMNU6ms+x7Z5hdTeX89s6GJWywmE5znX88IP366CDPHRmkbzROtc915gsnlkECGDwCb/4QDj0O264zOymLVUMClCgOWsPBn5r5pwt+32xol6c3O0eIJjNcuKFiERso5mpHY4BXTozw/NFBrjt3cp3pGVRshPVXwNGnzRYday9clDaK4iQBShSHzldMj2nrtbMKTrne07qKEkmIKFIuu5WdawP596ImW3sRhLrh9Qeh9UeQjMgi3lVC5qBE4cVG4OhTYzXZzpnVpdJ7Wh52NAZw2a08f3Rw9hcrBWXroed16HoNSsplEe8qIQFKFJbWpv4amKrWs5hjkN7T8pHrRR3tj9A3Gp/9Dd7aCzVnm4Xb/YfAWSqLeFcBCVCiMHLljB64yVSKCKw12VuzIL2n5WVevahgu8nsrNoKqRj0vGkqTsgi3hVNApRYerlyRuE+M59gscPhX8xquEZ6T8vPvHpRuUW87jKo3maCVMc+U3lCrFiSJCGWXtuj4PRDqBPQULcD0gnz+AyT3rmK5W90jBBJZvjkezYuTZvFgtjRGGBvaw9//pNWvC479QH3zJXOwSREPPfP5nOXH/xrzYJunTWbHbrLFr/xYslJD0osvWA7RPrML5byDWZeIY9yRrmK5cORJMm0qZT98KtdUrF8GTk2EOZIf5ieYByfy5Z/pfOJi3hHOyHQCFf/bygpg1d/AJE5DBuKoic9KLG0tIZMEkZOQNVZZm0L5FXOKFexPJJIk87C9jUesto8LhUjloe9rb2sqyjhaH+EzpEYW2u944/nVWlicg+7Zju8/gA8+xVTHis2JCnoK4j0oMTSOvECOLzgKAWnb2yIJr9yRp0jMTSa9uEoZSV2vC67VCxfZjpHYgRKHNSXuRmJpugOxuf3NfRUw5pdcPw5aP+1+Z6SFPQVQwKUWDpdr5qKABuvhN/4OzNvMNpphm0uuXPGv3grSx20dgZx2ixsrPYAUrF8uclVOq/zuygvdXB8MErHcHR+X8N3noE155kF3r37TQ/d5ZcU9BVAhvjE0ug7CG89Yeactn7QlDSqOzvvyxPpDFYLxFJZzqp2Y1FqvGL5DedL/b3lYndLDfc8cwyADVWljESTHOoNc/2ueWypEWwHX71ZF9V/EAYOmV2Y4zI3udxJgBKLp6fV/BXb32YKwa67FLb/lglOs5DNava29uCw2bj76rN4vSNI50iM+oCbG85vkPmnZSRX6Xxvay+dIzHObQyQymQ50B1iR2OAEsccfiX5G82wnjtgKpGMdkDfIZN4M/yOqZAvliUJUGJx5NY6WewQGQI0DB2BgbdmPXn9y8MDHO2P8N6t1exoDPDeZtnjaTmbWOkcoCcY54f72nns9W5+e2c9NussZx5OSUH3mTlOf70pnfX6g1BSZbbvGO2UBIplRuagxOJoexSyGQgeB5sDGs8Hd8Ws5wX2dwV5+fgwOxr97GicXaUJsTzU+l28f3stnSMxnjzYh9Z6ljeYlILuDsAVfwrv+Ty4y+Hlf4UTz5khQEmgWFakByUWXjoBx39lApTLD1VbwOoElz2v0jS5xbhv94UYCie5dFMFV5wlFQNWsi21XgYjCR57vYv/butFa/JfxAtTp6ADRAehpsX0oHreAG+tSabIY1G4KDwJUGJhBTvMD7/WZqK6uvlkAdg81jrlFuO6bRZGokmyaA73R3ird1Tmmla4gNvO4b4wiXSWsxv844t457UzcrDd1Hn0rTHzUaPdQDcMHYNEyASr3FxpsF2GAIuMBCgxP7kf7pET5v82N1RugnffBW88ZDKpXD4TnOIjZnfcM9jb2gto2kdigOK8xgCJdFYW464CT+zvZWudj47hKEf6wjSWl+Bz2eb3tZ+YQFF5lqlA0XfQ1IB84VtmqcOJ58w8la/+5BBgHssexOKTACXmLpcIYXebLL1Iv6kwfd7N0LATPDWn/mU6w9bt0WSaF48NkslqSp02ttR4cdmtOGwWWYy7CnSOxKjzu/A4fRwdiHB8MEqpw8pQJDn3m05OoEglTFC67H+aUlvPf8MMSSsrWO0nK+rLEGBRkAAl5u6NB00PabTTDOPV7QCrA95+wgSo6eYFODnPZNLFXWyt9XF8KIrWUF7q4KwaL2psaFAW464O9QE3wVgKv9vOlhoPg+EkB3tGiSYzPH9kEJ/Lxs8O9I4vMchrfiqXQDHdH0qtPzZzpaNd5vvYUQolFRAbMMdl+K+g1KwzZpbArl279L59+wrdDJFzyg9pA9SdB4lR85epy296SoEGM7yns+YH/Te/Me3tcvNMfrcdp03R1h2iL5Tg/dtquHhjBQ/t68DvNmWMQvE0wVhqfvMQYlmY+H2R+9oPR5K01Ps4NhDhQNcoZ9V6qfG5Fu774qkvmWE9RwlEBswoQLjPjASsuwy6XjZzWCWVJ4epc8N/ErwWjFLqZa31rsmPSw9KGNP9sOWG8ZxeUBZ451dw8Kew/gqoHdvhtLTq5H0mJEKc2ks6+Rfv3tZeSuwWosk0b/XG0FqzoaoUgIs3VhIosZ9ynSzGXR0mL+Kd+LX/i5+0YrdZOD4YJZHOUu114nefnJ+a7ntt5iedMATorQV7ifme3vQ+OPAwRIchFQd7u/ljDAVvPmSSgJ7/F5O6PtXclQSvBSE9qNVkpiDkCkxIaBiGc26AN/8DQt2QTYPOgMNjCnKWr4dt1zHyi69xNGxjIOWk0p5ggydN4Kq7aNONp/013B+K854t1TzwUjtWBUopfG47GypLcdgsdAfjfOWjOwr9LokidPcPX6fK66BjKEZfKAGA06bQKG5793p+/GrXtL3uGYPXdD8XD/+xma+KDpkRiwV8awAAIABJREFUg8QoZNIm+6+00lRE8daZYUGbG1JRc34u6J3y8yQ9rzOZVw9KKbUb+EfACnxHa/3lScfV2PFrgSjwu1rrV/K5dqEdbX2B3l//CGuog4y3gZoLP8KGlotWxbEzHu9pPTWY9B5jQ9fXCFx1Fxx4hKGklXf6h8lE36HMGmONLURJ8KsQHyVsr6Aj7qYv68VeEmCTt4Sq0U7adCOPpj7A+fqX1Kk+BnQ196QuY49uZG9rDx6HFYuCrpEYQ5EkQ5EkD73cMTa0Z6GxvAS33ZQ9CsZSMs8kppWbn9pQ5aGxvIThSJKOkRipdJZ/fuoIOqvJBFxorSlxWAHbWEYo438o1fldU6aut+lG9qY/TGcqRn3azW5dQzOAv5HBwV7eCpYSijvwOmvZ6glRVumEoWOEE2mG+/abvclsVspLnZRaM9B/kNFonGO9KYKpAdwuNxsCDirGFqlP+3NY21JUv0/m83tooczYg1JKWf9/9u48Tq6yTPT476lT+9JVvaXT6YQkQIQsw2ZQNgEFNKARd0ABxxkngyPojHKvy+jozOg48/k46tVhdMIgiHBh0FEvMExQEQYQ0ARBDGmWAIF0ekmnu6u6uvblvX+c6k7RdKcr6aWWfr6fT39SdZY6T51011Pve57zvsBzwAVAD7AduMwYs6tsm4uAa7AT1BuB/2OMeWMl+07lSFtQL+58jAM//2eK7jBFTwhHJo4jG6PtrZ8GaOh1R2847TXv38rEcGZGaD1nC6E9v+CFV3oQlw+fI4/JpXFmoyxvCeBMHWBfwsJpObAcQkq8jBo/r2t1kokcy66X+zGeMG6ng2y+iGRiHLdqOTe5LmMklcXvdpLJFUnnC4wkMjjEwUA8jd9lTRQ6hLxOIn4XuYLhqnOOfk3rSq8zqUOZ6vpULJXjj89Yyb/c/wKW2F9yiqWPM8GQyRs6wh6KRUNrwIPHZeF0CIlMnuaAm09d8Dqe3x+f9nfRM9Q97d9b8KX/pvvFvTjcfgKSxeSSuDLDrGwL4koM0Bs3OC3BcgiFoiFfKNIVLJL3trJvOE7BHcZhucgWBZNPsbJrGWPLz2HksVvIu8MU3U2QS+LIjtH8lk+AODjwq+9QcEeq/llzqHVHmqSma0FVkqBOB75sjHlb6fnnAIwxXyvb5t+AB4wxt5WePwucC6yaad+pHGmCevSGa5FUFGdmBDE5AKx8krzTD4Azn6RQejzn68zBdePbWlPsN3ld6RZWHPkkBaev9JqpiceCwcqnKDi9ZcfzIRjEGPu55cF63QWY5+7FWUhhLA8OUwAMVjFD3uHBX0yQMD4cpXHOCuIkXXTic2QYsdqhkCPpaiEtPgpi4crFyTpD7PCdxfmxO0hbIRL48RYTeAtx/iv4PrqLRxH0HExCAG5LSOeLdIa95IuGtqCHoMeJy3JMVGj91QWvO/JrBmrRmu535pu/eI5YKjeRYNK5AkNjGRwOYf9oBrdTgIO/o8YYxjIFLljXwW9eHCJfLBLwOBERBEjnigTcFgi0JXZzZvYRWgoDDFsdPOo5g+HQGrrSL/Dm4TvIOkOkHAF8xQTuQpwHWi7hpMRDeLIxcHpwmhxOk8eTi2IsD5HCARLGi8dRxGHyOChiCnkCpEg6AjiLGQoOz0Ss43+/QGmdFyN216ajkCFv+SYeF532YwNY+TT50nNnIVX6vBpfl5r4/LI/W8o/o8rXTf78SpEvfS5Z+RRFpw8QMs3HIulRjC/C6X/69SP6v51NF18XUD4+TQ92K2mmbboq3Hc8wC3AFoCjjjqqgrBey4r3kA90YiX3YRXteydM0RBK9wGQdoRw5RMT28/LOmMIZsrWFex1xl75qnXOwsH9MIZAZmBinVVIlXKeUEDwZg5gENJWCAoF7F82ISMBvLk4/SZCaz7LqNWGEQdFsSiKRcEhePNRnnOspMmRJWE1kRMXRiz8hTjDRT+Puc7k/eanFI0hjxAoxvEzxn9aF7BXVuJqvYRTkg/Tmd9P1NPB46F3MsxRnNjkJZUtEPG7cTsdeJwO4uk8YZ9rYloFh9jfIidPjTF5wFClZjLd70z5FB4hr/2RFvS6JgouosksPrdFJlckXzTEUjl8boszj23jd6+M0O5zUzRQNPbfqcsSRpI5jAHjP4af+Y8t/S0aikVDbDTDgFlOOvQ+NqZ+TWtugCGrgx3Bt/F0ehmDxTfw/uJPSeSFqPjxmyQibn7kfBenFR6hxZkk6QhOxB9wjBIrelhS3E/S1YIlBqGIwxQRR4FgfgihSMpagkNAKNrp1lHAn48iGNJWGDFSSkGAWPiyMTvhWiGkYIAiAEWceDNDAGSsEBTsa3oCFBD8mf0ApB1BHIXURGo3ZZ9RGUcQq3Dw3sSiJ4Qz3jO7/+ApVJKgZIplk5td021Tyb72QmO2AlvBbkFVENdrFELLcaSipFvXHwwsFSVZuvlOUlGML1KX69KHWDfmi/CW92zh0RueQ0p3zVvYF/3cqSjGt4KRljezYu/NhDwOMpYPT2EMRyZN/4oPsDK4ht8PhTg58RCdmT7ink6eCLyH5a1rWQ7EUk087zuZ50vHTKRynFCWhICJ5DSehA5VkaXUXJrpd23rgy8hIhMtLMsh/PEZK1nbGeaUo5onWvbjYqkcaztdE48nrxt/HktF2ON7PXtK61ypHOf4XMASnhpq4uTEQ3SV/T2tal1LbqyFjr03U/R4yFjB0t9hlv4Vl+Ievh/PFH/fBZ/9pc49xbqU73UTj4uT1sV96yceT94vc4SfNalp1jnSUQqhuZ+XrZIE1QOUD6C2HOitcBt3BfvOmY43vs/uG4VX942e+1GAhl93qPcfaV3LXbE0p6Yfpi0zwAFrCdub3sbm084CYOuDSfa3H/vqfvgNHaV1B7+dHk4S0laSWijT/a7N9Ds6ufVV/vsN0//uz7xuur+njmn/Dj1DrTX1eXKk6+ZSJdegnNiFDucB+7ALHT5ojHm6bJu3A1dzsEji28aYN1Sy71RmU2ZeS9UsNVXFx/R9+LNZp1S9m4+/iyNdV0ufJwtZxXfERRKlnS8CvoXda/R9Y8xXReQqAGPM90pl5v8CbMIuM/+IMWbHdPvOdDy9D0oppRaPWSWohaYJSimlFo/pEpTOqKuUUqomaYJSSilVkzRBKaWUqkmaoJRSStUkTVBKKaVqUk1W8YnIIPDyLF+mDTgwB+E0Ij03U9PzMj09N9PTczO1wzkvK40x7ZMX1mSCmgsismOqskWl52Y6el6mp+dmenpupjYX50W7+JRSStUkTVBKKaVqUiMnqK3VDqCG6bmZmp6X6em5mZ6em6nN+rw07DUopZRS9a2RW1BKKaXqmCYopZRSNanhEpSIbBKRZ0Vkt4h8ttrxVJOIfF9E9ovIzrJlLSLyCxF5vvRvczVjrBYRWSEi94tIt4g8LSKfLC1f1OdHRLwi8lsR+X3pvPxtafmiPi/lRMQSkSdE5O7Scz03gIjsEZE/iMiTIjI+3dKszk1DJSgRsYDrgAuBdcBlIrKuulFV1U3Yc3SV+yxwnzFmDXBf6flilAc+bYxZC5wGfLz0u7LYz08GeIsx5kTgJGCTiJyGnpdynwS6y57ruTnozcaYk8ruf5rVuWmoBAW8AdhtjHnRGJMFbgcurnJMVWOMeRAYnrT4YuAHpcc/AN61oEHVCGNMnzHmd6XHcewPnC4W+fkxtrHSU1fpx7DIz8s4EVkOvB3497LFem6mN6tz02gJqgvYW/a8p7RMHdRhjOkD+0MaWFLleKpORFYBJwO/Qc/PeBfWk8B+4BfGGD0vB30L+N9AsWyZnhubAX4uIo+LyJbSslmdG+ccB1htMsUyraNX0xKRIPCfwF8aY0ZFpvoVWlyMMQXgJBGJAD8VkQ3VjqkWiMg7gP3GmMdF5Nxqx1ODzjTG9IrIEuAXIvLMbF+w0VpQPcCKsufLgd4qxVKrBkSkE6D07/4qx1M1IuLCTk63GmN+Ulqs56fEGBMFHsC+jqnnBc4E3ikie7AvH7xFRG5Bzw0Axpje0r/7gZ9iX3KZ1blptAS1HVgjIqtFxA1cCtxZ5ZhqzZ3Ah0uPPwz8vyrGUjViN5VuALqNMd8oW7Woz4+ItJdaToiIDzgfeIZFfl4AjDGfM8YsN8aswv5s+ZUx5nL03CAiAREJjT8G3grsZJbnpuFGkhCRi7D7iS3g+8aYr1Y5pKoRkduAc7GHvR8AvgT8DLgDOAp4BXi/MWZyIUXDE5GzgIeAP3DwesLnsa9DLdrzIyInYF/MtrC/wN5hjPk7EWllEZ+XyUpdfNcaY96h5wZE5GjsVhPYl47+rzHmq7M9Nw2XoJRSSjWGRuviU0op1SA0QSmllKpJmqCUUkrVJE1QSimlapImKKWUUjVJE5RSSqmapAlKKaVUTdIEpZRSqiZpglJKKVWTNEEppZSqSZqglFJK1SRNUEoppWqSJiillFI1SROUUnVCRG4Ska9UOw6lFoomKKUmEZGxST8FEflOad0DIpIuW/dsteMdJyIni8ivRSQpIr8VkaPm4DVbROSnIpIQkZdF5IPTbOcRkRtK28RF5AkRuXC2x1eLmyYopSYxxgTHf4AOIAX8qGyTq8u2Oa46Ub6aiCwH7gH+CWgFXgS+MAcvfR2QxT4PHwK+KyLrp9jOCewFzgHCwBeBO0Rk1RzEoBYpTVCqbonIVSLyXyJynYgcEJFeEblgjg/zPmA/9uy7h01EPisiL5RaFbtE5N1l6/aIyLUi8pSIxETkP0TEW7b+ZBH5XWnf/wC8Ux7E9s/A9caYO40xKeB24NQjibns+AHgvcAXjTFjxpiHsafwvmLytsaYhDHmy8aYPcaYojHmbuAl4PWziUEtbpqgVD07ATgd+0NzCfBvwGfKNxCRu0UkOs3P3RUc48PAzebVU09/rZQQf12a+vtQXgDehN2q+FvgFhHpLFv/AWATsLr0fv64FLcb+BnwQ6AFuwX33qkOICJNwMXAv5ctdgDpKbY9nPPxOqBgjHmubNnvgalaUJOP01Ha/+mZtlVqOs5qB6DULJwA/KMx5l4AEdmFnQwmGGPecaQvXrqGcw7wp2WLPwPswu72uhS4S0ROMsa8MNVrGGPKuwb/Q0Q+B7wB+H+lZd82xvSWjncXcFJp+WmAC/hWKTn+WEQ+NU2o55W2fUpExpd5yo5RHs/hnI8gEJu0LAaEDrWTiLiAW4EfGGOeOYzjKfUq2oJS9eyPgLvKnm/ATh5z5UrgYWPMS+MLjDG/McbEjTEZY8wPgF8DF033AiJypYg8Od5KKcXYVrZJf9njJHZSAFgG7JvUcnt5msOsAu40xkTGf4D7gW2Vvc1pjQFNk5Y1AfHpdhARB3arLwtcPcvjq0VOE5SqSyKyGrsHoLyK7mTgyUnb/fcUVXnjP/89w2GuBH4wwzYGkKlWiMhK4HrsD+rWUuLYOd32k/QBXVLWJAKmq8rzYCe38eOuBjZid31OjulwzsdzgFNE1pQtO5Fpuu1Ksd6AXVDxXmNMbob3qNQhaYJS9eoE4A/GmGLZspOxr5FMMMZcWF6VN+ln2jJoETkD6KKsek9EIiLyNhHxiohTRD4EnA3cO83LBLAT2GBp/49gt6Aq8SiQBz5ROtZ7sLsGp7IdOEdElonICuD/An9tjBmevOHhnA9jTAL4CfB3IhIQkTOxr3X9cJo4vgusBTaXCjWUmhVNUKpenUBZa0lEWoGl2C2UufBh4CfGmPLuLBfwFeyEcwC4BniXMWbKe6GMMbuwq+seBQawuyR/XcnBjTFZ4D3YRRMjwCXYyWIqv8Lu6nwOeBj4oTHm+kqOU4G/AHzYlYy3AR8zxky0oEotss+XWot/jn0Nrb+sVfahOYpDLULy6i5upZRSqjZoC0oppVRNqihBicgmEXlWRHaLyGenWH+8iDwqIhkRubZs+QoRuV9EukXkaRH55FwGr5RSqnHN2MUnIhZ23/YFQA/2BdnLSv3r49ssAVYC7wJGjDFfLy3vBDqNMb8TkRDwOHaf/VyWAiullGpAlbSg3gDsNsa8WLpwezt2Jc8EY8x+Y8x2IDdpeZ8x5nelx3GgG7sySimllDqkSkaS6MIeBHJcD/DGwz1QadDIk4HfTLN+C7AFIBAIvP74448/3EMopZSqQ48//vgBY0z75OWVJKipbio8rNI/EQkC/wn8pTFmdKptjDFbga0AGzduNDt27DicQyillKpTIjLlKCmVdPH1ACvKni8Heg/jwC7s5HSrMWa6+ziUUkqpV6kkQW0H1ojI6tIIy5cyxRAqUykb+qTbGPONIw9TKaXUYjNjF58xJi8iV2MP52IB3zfGPC0iV5XWf09ElgI7sAeSLIrIXwLrsO/2vwL4g4iM3/X/eWPMPfPwXpRSSjWQiqbbKCWUeyYt+17Z437srr/JHqaygTGVUkqpV9H5oJRSi0oul6Onp4d0+jXzOap5YlkWkUiEtrY2HI7KBzDSBKWUWlR6enoIhUKsWrWKV89mouaDMYZcLsfAwAA9PT0cddR0s8a8liaoRaK7L8a2nQPsi6boivjYtKGDtZ3haoel1IJLp9OanBaQiOB2u+nq6uLZZ6cc+H9aOljsItDdF2Prgy8RS+XoDHuIpXJsffAluvsmz+at1OKgyWnhHU7X3sQ+8xCHqjHbdg7gcznoGUnS3Rcn7HMS9rnYtnOg2qEppdS0NEE1oO6+GN/8xXNc+6Pf881fPMeOPcPsGUoQT+eJp/MkswVCXif7ojrpqVK1ZNWqVfzyl798zfKHHnqI4447bsHjOZzj3nTTTZx11llzenxNUA2mvDtvaZOHZ/pHeaY/zlgqx/plTQgwNJYlns7TFfFVO1ylVAXe9KY3Hfb1m3o+7jgtkmgw23YOEPa5CHqcPDcQJ5bKcXS7n3iqQNFAk8/J3pEky/FzyalT3bqmlJqKFhotPG1BNZh90RQhr5OhsQyxVI7VbQFOO7qN5S0+wj4XRWNfIH7PyV36x6VUhV5daOSd10Kj7du3s27dOpqbm/nIRz5COp3mgQceYPnyg18oV61axde//nVOOOEEwuEwl1xyyavu67r++us59thjaWlp4Z3vfCe9vQeHTxUR/vVf/5U1a9YQCoX44he/yAsvvMDpp59OU1MTH/jAB8hmswCvOe4//uM/cswxxxAKhVi3bh0//elP5/z9l9MWVIPpiviIpXLEM3lcltDR5CGWyrN+WZi/uuB1pHMFtj74IsVqB6pUDXjg2f0MxjMzbvfzp/tJZQuMpQ9OeZfKFvjGz5/jreuXHnLf9pCHc49bUnFMt956K/feey+BQIDNmzfzla98hfPPP/81291xxx1s27YNr9fLmWeeyU033cRVV13Fr371Kz73uc/x85//nPXr13Pttddy6aWX8uCDD07su23bNh5//HH27t3LKaecwiOPPMKtt95Ka2srp59+Orfddhsf/vCHX3PMY445hoceeoilS5fyox/9iMsvv5zdu3fT2dlZ8fs7HNqCajCbNnQQS+XYH08T8DiJpfLEUjk2begAwOuyWNUW4PmBODPNpqyUso0kc3hdr/649LocjCRz0+xx5K6++mpWrFhBS0sLf/3Xf81tt9025Xaf+MQnWLZsGS0tLWzevJknn7SHO7311lv5kz/5E0455RQ8Hg9f+9rXePTRR9mzZ8/Evp/5zGdoampi/fr1bNiwgbe+9a0cffTRhMNhLrzwQp544okpj/n+97+fZcuW4XA4uOSSS1izZg2//e1v5/wcjNMWVINZ2xnmw6ev5B/++xmy+SJhn4tLTl3+qu6813UEeWH/GPuiKZY3+6sYrVLVVWnLpmckRSyVI+xzTSyLpXKs97l4/8YVh9jz8K1YcfD1Vq5c+aruuXJLlx5sufn9/ontent7OeWUUybWBYNBWltb2bdvH6tWrQKgo6NjYr3P53vN8/7+/imPefPNN/ONb3xjItmNjY1x4MCBw3uDh0ETVAMK+VycfnQr79+4fMoEdHRbEJclPDcQ1wSlVAU2behg64MvARDyOomn7Z6J+Sg02rv34ATmr7zyCsuWLTus/ZctW8bLLx+c/y+RSDA0NERXV9es4nr55Zf5sz/7M+677z5OP/10LMvipJNOmteeGO3ia0B90TQOETqavFOudzsdrG4L8vzAGMWidvMpNZO1nWG2nL2asM9FXyxN2Odiy9mr56XQ6LrrrqOnp4fh4WH+4R/+gUsuueSw9v/gBz/IjTfeyJNPPkkmk+Hzn/88b3zjGydaT0cqkUggIrS32zOz33jjjezcuXNWrzkTbUE1oN5YiiVNHlzW9N8/jlsa5LmBOHtHkqxsDSxgdErVp7Wd4QWpfP3gBz/IW9/6Vnp7e7n44ov5whe+cFjXec477zz+/u//nve+972MjIxwxhlncPvtt886rnXr1vHpT3+a008/HYfDwZVXXsmZZ54569c9FKnFC+UbN240O3bsqHYYdalQNHz3gd1s6Aofsn89Xyjy93fvYiiRxeuy9L4OtWh0d3ezdu3aaoexKE137kXkcWPMxsnLtYuvwRwYy5ArGJbNMErE8/vjPNMfpzeaKpWi6wCySqnaogmqwfSWxtfrDE99/Wnctp0DdEW8uCwH8XSesM+lA8gqpWqKJqgG0xdLE/I6CXldh9xuXzTFsogPyyFEk/Zd4zqArFKqlmiCajC9pcQzk66Ij7FMAb/bIpktAOgAskqpmqIJqoHE0zni6fyM3XtwcMQJYwzJTJ5YKvuqESeUUqraNEE1kL6YPVhkJS2o8fs62kJeoqk8Prdz3u7rUEqpI6H3QTWQ3mgKlyW0BT0Vbb+2M8zVbz6WHz/ew3tO6dL7oZRSNUVbUA2kL5ZmSZMXyyEV7zOezA6MzTyis1JKLSRNUA0iVyiyfzTDsvDhFTn43BYBj8WBsew8RaaUmk/r16/ngQceqHYY80K7+BrE/niGojF0RrzQvxO674LYXgivgLWb7Y0mL1u6AYDWgIchTVBKVd2qVasYGBjAsiwCgQAXXXQR3/nOdwgGg9Pu8/TTTy9ghAtLW1ANoLsvxrd+8Ry/2DXA/zx4P9H7vgGpKATaYWwAfvE39s/YAISW2use+Y6dyIDWoJvhREYHjlXqUPp3wv1fg5/9hf1v//wMlHrXXXcxNjbG7373O7Zv385XvvKVeTlOPdAEVefGp6IeiKdpC7pYPfgrfn8AhhMp6NkOB56DgV2wf5f9uO8p8IXBG7FbVNjXoXIFw2h67idfU6oh9O+0v9SlotDU9ZovefOhq6uLCy+8kJ07d3LnnXeyfv16IpEI5557Lt3d3RPbrVq1il/+8pcA/Pa3v2Xjxo00NTXR0dHBpz71KQDS6TSXX345ra2tRCIRTj31VAYG7FFjent7eec730lLSwvHHnss119//cRrf/nLX+YDH/gAV155JaFQiPXr17OQ46RqF1+d27ZzgLDPyWA8TcTvpj2znwPuNg70vUJLxA3NKyHeDwb7D2t0H6RHwdtkd/dRXiiRJeJ3V/HdKLXAnv+l3bMwk2fuhmwSMqMwPlxlNgn3fxWOf8eh9w12wJrXTtk+k71793LPPfdwwgkncNlll/Gzn/2Mc889l29+85ts3ryZXbt24Xa/+u/1k5/8JJ/85Ce54oorGBsbm5gO4wc/+AGxWIy9e/fi8Xh48skn8fns69WXXXYZ69evp7e3l2eeeYYLLriAo48+mvPOOw+AO++8k5/85CfceOONfOELX+Dqq6/mscceO+z3cyQqakGJyCYReVZEdovIZ6dYf7yIPCoiGRG59nD2VbOzL5rC67LIFQx+t0Xc00mYOI70iN3FF1xqt5Z8ETtZOSwY67eTVNieubMlYP+SayWfUtNIDoNrUgGSy2cvn2Pvete7iEQinHXWWZxzzjmsW7eOt7/97VxwwQW4XC6uvfZaUqkUjzzyyGv2dblc7N69mwMHDhAMBjnttNMmlg8NDbF7924sy+L1r389TU1N7N27l4cffph/+qd/wuv1ctJJJ/HRj36UH/7whxOvedZZZ3HRRRdhWRZXXHEFv//97+f8PU9nxhaUiFjAdcAFQA+wXUTuNMbsKttsGPgE8K4j2FfNQlfER280CYDXZfF8y5s57aXv4HdkIdBWuhbVBgik4+Bvh+gr4HDCKVcA9gSGYZ9LCyXU4lNpyyb6iv235IscXJaKQueJcPKH5jSkn/3sZ5x//sG4Pvaxj7Fy5cqJ5w6HgxUrVrBv377X7HvDDTfwN3/zNxx//PGsXr2aL33pS7zjHe/giiuuYO/evVx66aVEo1Euv/xyvvrVr9Lb20tLSwuhUGjiNVauXPmqbrzJU8un02ny+TxO5/x3wFXSgnoDsNsY86IxJgvcDlxcvoExZr8xZjsw+SLGjPuq2dm0oYPhRI50roDb6eBFaxW7XccRal4C6Zj9B3Xe38B5Xyz9cRXB6YbXbZqo4gO7UGIooS0opaa0djOko3ZSMkX733T0YIXsPJo8hbsxhr179045hfuaNWu47bbb2L9/P5/5zGd43/veRyKRwOVy8aUvfYldu3bxyCOPcPfdd3PzzTezbNkyhoeHicfjE6/xyiuvzHp6+LlSSYLqAvaWPe8pLatExfuKyBYR2SEiOwYHByt8ebW2M8wF6zrwuixGElmWuFKcuipC6PzPwLv+Fd78OTsRLd1gP37fjbD+3ZBNvOp12oIeRhI58oVild6JUjVs6QY44xr7S97oPvvfM6551Ze8+fKBD3yA//qv/+K+++4jl8vxz//8z3g8Hs4444zXbHvLLbcwODiIw+EgErFbe5Zlcf/99/OHP/yBQqFAU1MTLpcLy7JYsWIFZ5xxBp/73OdIp9M89dRT3HDDDXzoQ3PbKjxSlbTRphqWoNJ65Ir3NcZsBbaCPaNuha+vgIjfxQXrOvjom46GPQ/DS17oWDf1xiJ2t8Tu+2BsEILtgJ2gisYwkszRHqpsqCSlFpXxL3oL7Lh8rP9rAAAgAElEQVTjjuOWW27hmmuuYd++fZx00kncddddrymQANi2bRuf+tSnSCaTrFy5kttvvx2v10t/fz9XXXUVPT09BINBLrnkEi6//HIAbrvtNq666iqWLVtGc3Mzf/u3f8sFF1yw0G9zSpUkqB5gRdnz5UBvha8/m31VhaLJnF19ZwwMPA2Ro8B7iEFfO9bDC/dD/+/hWLuvuzVo/7IPJTKaoJSqkj179ky5/N3vfjfvfve7Z9znlltumXKbyy67jMsuu2zKdcuXL+fuu++ect2Xv/zlVz1ftWoVxixc+6GSLr7twBoRWS0ibuBS4M4KX382+6oKxVI5Ij4XxPvsqqKO9YfewR2AtmPtZFa054Jq9rtxiHAgroUSSqnaMGMLyhiTF5GrgXsBC/i+MeZpEbmqtP57IrIU2AE0AUUR+UtgnTFmdKp95+vNLEaZfIFktkDE74KBp+zqvPbjZ95x6Ykw+BwM7Yb247AcQkvApYUSSqmaUVGdoDHmHuCeScu+V/a4H7v7rqJ91dyJJe3CybDHAft2Qesx4Jp5wkJajoZMHO77e/um3fAKVjlP57mxlTPvq5RSC0CHOqpz0VSO1rHn6frN38FTP4KXH6ls+JX9peGPRvsm7pc6/qWbcA3uIpMvzH/gSik1A01QdS677yk29t6CJ/4y+JoBqWyMsO67ILISXB5IDIEvgivYwprh+xlO6HUo1dgW8kK/shWLh38LiyaoOufbfQ8FdwirkIHgEvC3vGog2GnF9trbu/z2Db2ANxAhlOnTESVUQ/N6vQwNDWmSWiDGGLLZLPv27SMQOLxZu3Ww2DrnGO3B5Qrad7ePl5aXDQQ7rfAK+254T6g0npjBUxgj6VtGQsfkUw1s+fLl9PT0oAMCLByn00k4HKatre3w9puneNQCGXYtoTPXY9dIukuTmpUNBDuttZvtrkCAQhbiA8Tice7Ovo1d9+/myVeibNrQwdrOQ9xPpVQdcrlcrF69utphqApoF18dyxWKPN10Lv7cCOSz9hh7lY4RNj50S1MXZOIMp/Jszb2dHs9qPJYQS+XY+uBLdPfFDv06Sik1TzRB1bHRVI6h4Bqyy0+zK/FGew9vjLClG+CCv4P17+J/2Eiy5Xjagh7yRfC7LcI+F9t2VjBXjlJKzQPt4qtj0VQOTBGfxwPH/ymsOYLxsxwOaFpGvvsFQs1nMD5UYipbIOR1si+amtuglVKqQtqCqmPRZA5vfhSvo2jP2nmkQp10OUdJpNJ4nRYA6XyBeDpPV8Q3w85KKTU/NEHVsdFUjubiEE5LZpegmrpYs8SPifeTyhXAGIbHssRSOTZtmMXrKqXULGiCqmPRVJZ2oojDWZo19wg1LaM96OXD651E/G7S+SKWJWw5e7VW8SmlqkavQdWxaDLHahOFQDs4rCN/IU8QvGFWuWL81QXnsbLVTzpX1OSklKoqbUHVqWLRMJrMESkOz657b1zTMnu6DiDscxFL5Wb/mkopNQuaoOpUPJ3HlY/jlyyE5iJBddk3+KZHifhdpHMF0jkdNFYpVT2aoOpUNJUlkD2Ax2lBcOnsX7Bpmf1vvI+wzwWgrSilVFVpgqpT0WQOf3YIr9uyB32drWCHfR1rdB9NmqCUUjVAE1SdiqVyNOWHcIfawXLN/gUtp52kRnsnWlDRpCYopVT1aIKqU9FUjlaGkdAcdO+Na+qCeB8eh+B3W9qCUkpVlSaoOpUYjRIkDXOaoDqhkIfEfiJ+reRTSlWXJqg6ZIwhP9qP1zVH15/GjRdKjO4j7HMRTerEhUqp6tEbdetQIlvAkxosJag5bEFFe+CVx+DlR1jXdCKDjtPJF1bhtPR7jFJq4eknTx2KJrP4c0O4gi3g8s7Ni/bvhEf/xa7ks9wECglev+8WEq/8fm5eXymlDpMmqDrT3Rfjuvt30/PKSzzYK3M3oWD3XeCN2Ne08ilc/hBpZ4jirjvn5vWVUuowaYKqI919MbY++BKjo2O0OhKMOFrmbtbb2F7wNk1MG+8rpshYQYrRvbN/baWUOgJ6DaqObNs5wBrzMseN/Ixjsk8yEs+SjoTYttM/+4Fdwyvs6eLdAQCchSS+Yo5Rz1Ja5yB2pZQ6XNqCqiOFvj9wzoHbCWUPkJEAYoqcc+B2Cn1/mP2Lr90M6Shkk+BwIolBQiR4qf0ts39tpZQ6Apqg6siZuUeJE6AIFC03KXcLcQKcmXt09i++dAOccQ34IlDIgsArx32UHvfRs39tpZQ6AtrFV0eO98X4TdJHSz5N3uMjkyuQNj7e6JujQomlG+yf5adC3xM4OtYT64lhjEFE5uYYSilVIW1B1ZHmzqM5qV3wSpaEceNxWZzaadHcOcetnEAbFPK0WknyRUMiq9NuKKUWXkUJSkQ2icizIrJbRD47xXoRkW+X1j8lIqeUrfsrEXlaRHaKyG0iMkc37ixCazcTKsZoc+fYcNQSTu+0aHWk7OtHcynQDkCziQLoiBJKqaqYMUGJiAVcB1wIrAMuE5F1kza7EFhT+tkCfLe0bxfwCWCjMWYDYAGXzln0i83SDQwe814KDg+u/Jh9veiMa+xuublUSlBNBTtB6Zh8SqlqqOQa1BuA3caYFwFE5HbgYmBX2TYXAzcbYwzwmIhERKSz7Bg+EckBfqB3zqJfhOKOCGPh17P04s9DsGV+DuJ0gy+CPz+CyDJiOu2GUqoKKuni6wLK79bsKS2bcRtjzD7g68ArQB8QM8b8fKqDiMgWEdkhIjsGBwcrjX/RyY0dwOl04vJH5vdAgXas5AFCXh3VXClVHZUkqKnKt0wl24hIM3brajWwDAiIyOVTHcQYs9UYs9EYs7G9vb2CsBanfGIIfM3gmOf6lkAbJIeJeEQTlFKqKir5lOsBVpQ9X85ru+mm2+Z84CVjzKAxJgf8BDjjyMNVJjGMIzBPXXvlAkvAFGm3EkQ1QSmlqqCSBLUdWCMiq0XEjV3kMHkE0TuBK0vVfKdhd+X1YXftnSYifrFvpDkP6J7D+BcVUyxgUiM4Awsw+FCpUKKFKKlsgUxeS82VUgtrxiIJY0xeRK4G7sWuwvu+MeZpEbmqtP57wD3ARcBuIAl8pLTuNyLyY+B3QB54Atg6H29kMcgkYphCHldoAbpA/S3gsIiYGNBOLJljSZM1/8dVSqmSikaSMMbcg52Eypd9r+yxAT4+zb5fAr40ixhVSSK6HwBfuG3+D+awwN9CKD8C2KXmS5r0Fjal1MLRkSTqSCpmVzf6w3M4zfuhBNrx54YBvRdKKbXwdCy+OpIZPUDB4SIYalqYAwbaib7wO57s72fHnmE2rmph04aO2U/toZRSFdAWVB3JjR0g44rg9yzM94oXkl4efzlKKB/F77aIpXJzN0GiUkrNQBNUHSmODSP+lgUbWfzePQav08EK9xiZfJGwz0XY52LbzoEFOb5SanHTBFUvCnmKqSjWQpSYl7wYt7DcHloYJZsvYowh5HWyL5pasBiUUouXJqh6kY6SzRdwhhYuQXU1+xkhTMREMUA2XySeztMV8S1YDEqpxUsTVJ0oJIbI5ou4QwtQYl6yaUMH+4tN+LLDmGKRwbEMsVSOTRs6FiwGpdTipQmqTqRigxjA37RwCWptZ5jzNq4n4spTyIzhthxsOXu1VvEppRaElpnXifToIDnLRzAYXNDjrl65ilXDbexOuFmzdpkmJ6XUgtEEVSeyowdIOcOEvK6FPfDYIPLyw7wl8WvSiRMh+MG5nyBRKaWmoF18dSI/NkTaGSa4QPdAAdC/E7b/OxQLiCtAMRWFR75jL1dKqXmmCaoe5NLk03GMrwW3cwH/y7rvAm8E/K14yTAmAft5910LF4NSatHSBFUPUiNk80Ws4MKVmAMQ2wveJnAH8JgMuXyRvDtoL1dKqXmmCaoepIbJ5Iu4FrDEHIDwCkiPgsuHUwxOkyWbiNrLlVJqnmmCqgfJYTJ5gze0ADPpllu7GdJRKBZwCvgzB8gno/ZypZSaZ5qg6kB27ABJR4CQf4HnY1q6Ac64BkIdWIUxjMNi7/F/qlV8SqkFoWXmdSBTKjFfttAl5mAno6UbcHgjvNLrw+c7lvULH4VSahHSBFXL+ndC9524dt3DKrOM5viboOP1VQlFAu200Eu/TlyolFog2sVXq/p32vccJQ6Qd/gQUyDy5L9V7x6kQBthM8poKlud4yulFh1NULVq/B4ky03BQNrditNfxXuQAm34HQVS8RGMMdWJQSm1qGiCqlXj9yDlUuSLBofbh3jD1bsHyd+Gx2XhzgyTyBaqE4NSalHRBFWrxu9ByqcoFMFye+3n1boHKdCGx+nAlxshptehlFILQBNUrRq/BykxRMY4CRST9vNq3YPk8uHxN+HLRYklNUEppeafJqhaVboHqSgOpJDG4Q/b9yRV8R4kT3gJ/twwo2lNUEqp+adl5jWsu7icgfQx/Kp4BjnO40qzgrVVjMcKLSHCLoaTWsmnlJp/2oKqUd19MW76n26ymRR5d5hMvsDWB1+iuy9WvaD8rfgdBZLxkerFoJRaNDRB1ahtOwfocKWwHELCaqIt6CHsc7Ft50D1ggq04XFa5Eb3Vy8GpdSioQmqRu2LpmizEhSKhoQjiNtpEfI62RdNVS8ofxselwOTOEC+UKxeHEqpRUETVI3qivgoJkfIFyHjDGE5hHg6T1fEV72g3H7cvhDe7AjxdL56cSilFoWKEpSIbBKRZ0Vkt4h8dor1IiLfLq1/SkROKVsXEZEfi8gzItItIqfP5RtoVJs2dGCSIwwXvDidTmKpHLFUjk0bOqoalzu8BL/eC6WUWgAzJigRsYDrgAuBdcBlIrJu0mYXAmtKP1uA75at+z/ANmPM8cCJQPccxN3w1naG2XSMh6TVRDJbJOxzseXs1aztDFc1Lm+4A19+hJhW8iml5lklZeZvAHYbY14EEJHbgYuBXWXbXAzcbOxB2h4rtZo6gQRwNvDHAMaYLKCfbBVa6k6zrLOTj/3RMZx73JJqhwOAN7wEl8kxFh8BmqsdjlKqgVXSxdcFlA8A11NaVsk2RwODwI0i8oSI/LuIBKY6iIhsEZEdIrJjcHCw4jfQsHJp8ukECQkRqsY8UNOQUiVfJqaVfEqp+VVJgpIplk0eznq6bZzAKcB3jTEnY7eoXnMNC8AYs9UYs9EYs7G9vb2CsBpcaoRsvkja2UTYV0P3Uwfa8LocZLXUXCk1zypJUD1A+Qily4HeCrfpAXqMMb8pLf8xdsJSM0mNkMkXSDubaqoFhTuAyxugOKatXKXU/KokQW0H1ojIahFxA5cCd07a5k7gylI132lAzBjTZ4zpB/aKyHGl7c7j1deu1HTSUTKlFlRTLSUo7CGPnOlh0jmddkMpNX9m7DsyxuRF5GrgXsACvm+MeVpEriqt/x5wD3ARsBtIAh8pe4lrgFtLye3FSevUdFIjJPBhudx4XbV1u9owIQYHdnHtHU9yzJIQmzZ0VL26UCnVeCq6uGGMuQc7CZUv+17ZYwN8fJp9nwQ2ziLGxSk1wpgjRJPPhchUl/iqo7svxhPPv8ypqe2sG9xHNL6cu/adBW87X5OUUmpO1dZXc3VQaoQYAULeGiqQALY/9jCnpX6NlxxjEiQsCd46egfbH3u42qEppRqMJqhaVMhBZoyoCdTc9aeO3p+T87WRd7hxFtJknE0U3GE6en9e7dCUUg1GE1QtSkUpFA0xE6ytCj6gSw4QpQnjcOEo2APXxo2PLjlQ5ciUUo1GE1QtmigxD9NUS/dAAUtWHItk42TEg6uQJJMrINk4S1YcW+3QlFINRhNULUqNlErMa2sUCYAlp76fE9vAbTlwZJOESHBim71cKaXmkiaoWpQaIY2bguWlqcaKJFi6gch5n6Jr+Uo6vRmOX9FO5LxPwdIN1Y5MKdVgNEHVotQICUcIhwgBd40lKIClGyi86X/xYus5DB7zfk1OSql5oQmqFqVGGJUgIa8Th6N27oEqF2pdCkAq2lflSJRSjUoTVK0pFiAzSsyEau4eqHIerx/jDZON9lc7FKVUg9IEVWvSMTCGEeOnyVdbBRKTSXAJhbgOGquUmh+aoGpNaoSiMYwUa28UiclcTR0Uk0NQyFc7FKVUA9IEVWtK80ClanAU88k8kaXk8gUyowPVDkUp1YA0QdWa1AhpY5Fz+Go+QQVa7EKJsWG9DqWUmnuaoGpJ/054/Ae4Xvg5p+29nkj82WpHdEih5g6KYpEc1ko+pdTc0wRVK/p3wiPfsVtQniV48qMEH/+uvbxGRQIeUq5mMjHt4lNKzT1NULWi+y7whEGEjLgxnggOX7O9vEa5LAfG30Z+VLv4lFJzTxNUrYjtBZcbTJE0HjxOB3ib7OU1zNnUQT4Zh2yy2qEopRqMJqhaEV4BCfueopRx4XE5ID1qL69hnnAHqVxhInallJormqBqxdrNkDiAyaVJFl34i3FIR+3lNczX0km+aEjriBJKqTmmCapWLN0Aa95GwRXAlx3C8rfAGdfU/ECsTU0R8g4vyRFNUEqpuVXbQxUsNp4gY2su5heps7n4pGXQHqx2RDNqDnjY7WrWFpRSas5pC6qWJIdJWE0ANT8O37iwz0XK3UJudACMqXY4SqkGogmqVuSzkIkTlxBAzY/DN85yCBJcQjaTtq+ZKaXUHNEEVStSwwBEpQmvy8LjtKocUOXckVIl35hW8iml5k59fE1fDJJDDI6l+fGeBC+k+knnCmza0MHaznC1I5uRL7KUdK6ISexH2l9X7XCUUg1CW1A14uWevTz+coy+jI8lIQ+xVI6tD75Ed1+s2qHNKBwMkHQEyUR1yCOl1NzRFlSNeOq5Pbi8IYxx4XVZhEtFEtt2DtR8K6rZ7yZeyGKeuAX23GffXLx2c82XyCulapu2oGpEZnQ/OU8zhaKxhznCLpTYF01VObKZtY49z1Gx31JMDkOoE1JRe+DbGh7oVilV+zRB1QJjWOpKsT/vB8Drsgsk4uk8XRFfNSOrSODFe0i5WsmLG/Jp8EXAG6npgW6VUrWvogQlIptE5FkR2S0in51ivYjIt0vrnxKRUyatt0TkCRG5e64CbyiZOMe1u+nNBkjnCridDmKpHLFUjk0bOqod3Ywcoz0U/O3kCgYyY/bCOhjoVilV22ZMUCJiAdcBFwLrgMtEZN2kzS4E1pR+tgDfnbT+k0D3rKNtVKlh2oNeTjzuaLwui5FEjrDPxZazV9f89ScAwivwWUVyxgHZUoKqg4FulVK1rZIiiTcAu40xLwKIyO3AxcCusm0uBm42xhjgMRGJiEinMaZPRJYDbwe+CnxqbsNvEMkhALxN7Wza4OIjZ66uckCHae1meP5r7B/LMvJSD+khi6ODeSKnXFHtyJRSdaySLr4uoLyvpqe0rNJtvgX8b6B4qIOIyBYR2SEiOwYHF9kNn8kRsJwMZt1E/PUxxFG5brOCm4qbGSJCixlitOhja+7tdBttQSmljlwlCUqmWDZ50LUptxGRdwD7jTGPz3QQY8xWY8xGY8zG9vb2CsJqIKlhjK+FaCpPxO+udjSHbdvOAVItx3N302U80/QmnlvxfpItx7Ntp94XpZQ6cpUkqB6g/KvwcqC3wm3OBN4pInuA24G3iMgtRxxto0oOkXE3k80Xaa7DBLUvmqIt5GHYaiVXKBLMDtZNibxSqnZVkqC2A2tEZLWIuIFLgTsnbXMncGWpmu80IGaM6TPGfM4Ys9wYs6q036+MMZfP5Ruoe4U8pGMTg8Q212EXX1fERzpXJO8MMGY8BLODdVMir5SqXTMmKGNMHrgauBe7Eu8OY8zTInKViFxV2uwe4EVgN3A98BfzFG/jSY2AMUTFnmYj4qu/FtSmDR3EUjkQB/tpxkr0102JvFKqdlU01JEx5h7sJFS+7Htljw3w8Rle4wHggcOOsNGVRjEfLgawHFI302yUW9sZZsvZq/m3/3mRl2JN/JHvJbac2VUfJfJKqZpVf5+GjaZUYn6gECTiB4djqnqT2re2M8yfn3MMj20/wEmeYfxBvf6klJodHeqo2pLD4AkynKEuK/jKtQU9JNztJLMFGO2rdjhKqTqnCarakkMYXzPRZI5InUzzPp2WgJuiK8Co8UNcE5RSanY0QVWTMZAaJumMkC+auiwxL2c5hOaAmyFHC8T7qx2OUqrOaYKqplwKcmni4xV8dVhiPll70E2/abarE3N6HUopdeQ0QVVTqUBihNI9UIH6bkGBfR1qkGbyxaJ28ymlZkWr+Kqlfydsvx56Hsc/MMTSwHkE3GuqHdWstYc8jLnbSWQKhOP90HJ0tUNSStUpbUFVQ/9Oe8bZxCB4w+QzSd7Ydwsy8HS1I5u1tqCHgsPDqCMEo5NHxFJKqcppgqqG7rvsGWfFApePuATB29wQM9D63RZ+t8WQaKGEUmp2NEFVQ2yvPeNsNkHR5SedK+LyhxtiBloRoS3oYb9phkzc/lFKqSOgCaoawivsIY7yaXKWDwP4TKJhZqBtC3nYV2jGGKOtKKXUEdMEVQ1rN9sf3Lk0aTx48qP4C2P28gbQFnTjygyTe/Eh+O/PwP1fs6+7KaXUYdAEVQ1LN8Dx7wSXl3wqRsbZhOPMT9jLG8DS1Auc0nc7uaIBpxdSUbsoRJOUUuowaJl5tXj88LpN/L71Ep7tH+O05Y1Tjh15eRuvOEMkXUIgmwBfaVTz7rsaJgkrpeaftqCqZWwAgh2MJHJE/C5E6nMU86lYoz04fGHGCEAxB9mkXRTSAEUgSqmFowmqGgp5SAxBqIORZLYuZ9E9pPAKmiRlDxoLkI5CerRhikCUUgtDE1Q1JA+AKZLztTOWydf9NBuvsXYzIcYo5FIUHB77ht10tGGKQJRSC0MTVDWUSq9HnS0YQ92PYv4aSzeQPvXjZJxNZPNZu5vvtKv1+pNS6rBokUQ1jO1nf7LIdx4+wOOvRElk8rz39Y01RXq/92i+mX8vx8leLuK3tCXd1P9Ig0qphaQtqCro3beH+3pgKJEl6LHI5gtsffAluvti1Q5tTnT3xbj1N3spFIskfMvI5Apse+ixhnl/SqmFoQlqoRnDiy+9RNG/BMshuJ0OmgMewj4X23YOVDu6ObFt5wBhn4uWgIdYwUPB38ZyBhvm/SmlFoYmqIWWGiGZSlIMLCGVK+B1WQCEvE72RRtjgr990RQhrxO/2yKZzRPzdNFe3E/fiI7Lp5SqnCaohTY2QMjror8YJpHJE/TYlwHj6TxdEV+Vg5sbXREf8XSeJq+TooE+xxLyuRzH+TRBKaUqpwlqoY0NcGxHiJfTfpLZAj63RSyVI5bKsWlDR7WjmxObNnQQS+XAgDGGZ1LNpPOG8zsbo4WolFoYmqAWWnyA9o4u3rahC4/Lsmee9bnYcvbqhqniW9sZZsvZq2kJesgVDFlxceK641jpOFDt0JRSdUTLzBfa2AC0rMZnnJx7XDsfO+eYhhrmaNzazjBrO8OcuCJMd1+cjhXHQ89vIZ8FZ4Pd96WUmhfaglpImTHIJiC4lIF4mo6QtyGTU7nlzX6y+SJDrk4wRR2PTylVMU1QC2nMLrPO+9s5EM+yNOytckDzb7zw45VCMzgsGNlT3YCUUnVDE9RCKiWoQcIUjaGjyVPlgOZfwOOkJeBmbywPTV0QfbnaISml6kRFCUpENonIsyKyW0Q+O8V6EZFvl9Y/JSKnlJavEJH7RaRbRJ4WkU/O9RuoK2MD4IswkLS79ZY0NX4LCmB5s4990RTFQgH+8J/wkz/XWXaVUjOaMUGJiAVcB1wIrAMuE5F1kza7EFhT+tkCfLe0PA982hizFjgN+PgU+y4eY/shuISB0TQBj0XIszhqVLqafYSiz5J9ZhvkUuD26yy7SqkZVfIJ+QZgtzHmRQARuR24GNhVts3FwM3GGAM8JiIREek0xvQBfQDGmLiIdANdk/ZtfP07YddP4dl7YekfkfGH6OhY3/AFEuOWN/sZGb6fWKAVrztuzw3Veqy9UmfZVUpNo5Iuvi6gvPSqp7TssLYRkVXAycBvpjqIiGwRkR0ismNwcLCCsOpE/067pRDrA0+IQrHAsS/cyKrcnmpHtmCCHiethQFGin7wNZfmwzI6y65S6pAqSVBTfc03h7ONiASB/wT+0hgzOtVBjDFbjTEbjTEb29vbKwirTnTfBd4IOBwgQsLdTtoKcdT+X1Y7sgVlRY4iMxbFBNqgkNNZdpVSM6okQfUA5Z8iy4HeSrcRERd2crrVGPOTIw+1TsX22i2FVBRcPsYKFhkrSDDTX+3IFpSseyeu3CipnAFx2OXmOsuuUuoQKklQ24E1IrJaRNzApcCdk7a5E7iyVM13GhAzxvSJfZHlBqDbGPONOY28XoRX2MkpHQVvhEQmT4gkruajqh3Zgmo79hR2LLucqIQAgXwaTvsLvf6klJrWjAnKGJMHrgbuBbqBO4wxT4vIVSJyVWmze4AXgd3A9cBflJafCVwBvEVEniz9XDTXb6Kmrd0M8V7IJsEbJjs2TFiSi67l0OR1kW9fx47VV8Hmb8FRp4OrMUZvV0rNj4rqnI0x92AnofJl3yt7bICPT7Hfw0x9fWrxWLoBjj0f/vBj8pkxRmnCnHwFKxZhy8EY8//Zu/PwuM7y4P/fe86s0ixaLcuy4yVxHC/ZjBOyAQGS1AmEdIEsNAmlpWlKE2gDVyGFlvIWSvu7WNpCCg2EUJY3YXkhdSA4IZA0QDY7K7IVJ44dW/tiSSPNvj2/P85IkWXZmrFGmhnr/lyXLmmes91nNPat55zn3A/ff/ogv97j5l3RKKtdO1jefGq5w1JKVSitJLEQRODMa+m++Es8edJNhFadXe6IFlxHb5jfvnqISDJDsMbDQedJvPDCs3R0nUAjNpVSJaUJar6lYjDeB/Wr6AsnAGgOnPgljqbb3t7P0qAHr8siksgQr1tHrZVj586nyx2aUqpCaYKab6MH7Gd+GlbTP56kvsY1Oc37YtI9GqfR72AOyiUAACAASURBVMHjdDAaTzPuWQreAI7BjnKHppSqUJqg5tvIawzEc3zpqQjf+PU+drw2TEdvuNxRLbiJaeAb/R7CsTTpnKHTuYrVjn67l6mUUtNogppPxtD32m62d3kZiqbxOh0Y4M7H9i+6JDUxDbzHKeSM4bVDUfayktP8Udj+cbjvg1pAVil1GE1Q8yk+wv6uXhKBk7AcgoiwNOgl5HOxvb2/3NEtqIlp4FuCPjI5iKeyvO/MGhrCHTBy0J6KQwvIKqWmWBzltMtleD/jiTTppSsZHUvhEHt+JLDvySw2E9PAX7yumUf3DNLS8/18YhqBXBp8dfaKWkBWKYX2oObXyH7c/gYG0j6Goynqa9w4RBhPZCZnml2M1i0N2O9D/36oX2U3jvXa37WArFIqTxPUfMllYfQAJ5+6kb6xJOOJDPW1LsLxNOF4mq2bWsodYdnUuJ2saqqhO9eIyaSgttGutpFLawFZpdQkTVDzZawHMilWnLyJ809upMZtEU/nCPlc3PTm1axvDZU7wrLa0BpkV+hiouOHwBOCbAYGX9YCskqpSXoPaj70tcMTX4H+dnLZNLWJc7jh/LO44vTWckdWMVY31fKL+nU877yBi9JPwPCrkByDi27T+09KKUB7UKU3MUHhWA8ElhEZD7PxwLfZZB0sd2QVxWk5WNcS4PlUG8k3/S28+267gGxGn4lSStk0QZVax/3g9tv3oHz1DGW8pFxB2vp+We7IKs761iA9o3E+vW0XH93ex896AwzsfgzSiXKHppSqAJqgSi3cCakIYDC1zQxHU9QG67HGusodWcUZjaVo7xnj4HCc1pCXl9ybeHZfP6+9+Fi5Q1NKVQBNUKUWbLNni/XVM5Z1k84aml1JHZk2gwd39bMs5CWdzZHO5nCElmG5vPDY/wc/vlkrSyi1yGmCKrUl6+0elCfAofE4vuw4AaI6Mm0G3aNxTmqsQQS6RuI0Rl7hpOTLSHLcnkVMK0sotahpgiolYyA+DGsvIxc6idRIJ7XBRqwLP6Qj02bQVucjlTG0hnwMjidZOfAwI45GUjVL7UEmbh946+z7ekqpRUcTVCmNvAaRQfa1XML/ibyL25J/wbe976XD6OW9mUwUkA14LFwOwYx2MWp8NCxfByZnPxflCWhlCaUWKU1QpdS1g/6kxR27PHQOxwj5nIgszurlhZgoIFtf68HncTJgNXNqMEdjfT00nmw/tDu4R+/fKbVI6YO6pRIdgkOv8r+RVdT6vPSNR2iodU/W39ve3r/oq0fMZKKArDGGh345SvqVb5KJ1eCsbYZwD4zsg803ljtMpVQZaIKaq752+x7JwSfAGHpZScSXJpMzLAl4AQh4nYuyenkxRISz33gR9/aPccq+R2nKtpPzL+eUpSfTtO8ROPg4RPrt3tT6K/WenlKLgCaouZioGjHxYK7Hz1sP/Q93jWZpat5A0OcCWPTVywt1KJLkiWgrv5KrecMp9WRz0DT4JH8W+wa+UCss3/z6yL4LbtUkpdQJTu9BzUXH/fYos3QMMJjGtWTdIc5N/Jb6Gjc5Y7R6eRG2t/dzSrOfWo+TA4diBL1ONqbbOcAyyCYg3AW+kI7sU2qR0AQ1F+FO+3mdcCfUNDKQsEhYft60JEGj30NvOKHVy4vQPRqnrtbNysYaxhMZXhmI0JQd4CDLwL8ERg/Cob3g8evIPqUWAb3ENxe1zUT3/pbBpMUrYwHG0kOs8adZcfKp/M1bTy13dFWnrc5HOJ6mOeAlmzO8dijGK8l6VtYkoelUsDx2YoqNQGi5XWki3Kn3pZQ6QWkP6nhlkgxGMwyPjjBkLSGZE9zpMZKRYfY3v63c0VWlieeiwvE0S4JemgIeHsqdgyM5yjN7XuOhHjcd0VqSQ/uge4c9aCLYphUnlDpBaQ+qGBMj9kYPQnKc7hE37W1/wbJoB7WxTlKBNp5tfBvJ3gb+Rv+YL9rEc1Hb2/vpHo1zSrMfT+v5/NczcFl2B2s9w/RZy4kks2xklJqR/SAOqG2yh/n//G/tKeS1R6XUCUETVKEmRux56yCbgrEe3FEXA/5GfmZdjXeJxeltIQzQq0PKj9vEc1ETvvSLl3G1nc73x06lxm2xoqGGPzjwWXZmT2b5cBembyduByyxxvC43LDyQh3pp9QJQhPUsUz0mMKdMPIaYUeI3u5DuOIDJHxLOWi8tPY8THDZn7Gm2Y+IMBZP65DyEuoejbOqsZb6GjcHDkXZ0zfOK8k6THyU3JJ11HnCrBh5ing6ypjLz0u/24uztoFT63w07rjLHlyh96mUqkoFJSgR2Qr8O2AB3zDG/Mu05ZJffgUQA/7EGPNsIduWw772J+l/6kdY411kA8tpeeO7AQ5rW75uM4EDD7Mv4mQkZbE5/AqObAq/dylDnpN4Od5APJlhhWuA1pAXp0Mm759cc87yMp/hiWNi4ER9jZs6n4uB8STbBt/AdWYb2dgow94gS3ESM14iWT+rMvtJjR5k4JDBRzcvhy5iIFNDU/9+1vR8kfGVl9C159nJ37Nr6QbSfbuP+VloeeO7WdPkf/2PldAKaF4Pgx2HJz84fB1NiErNiRhjjr2CiAW8DFwKdAE7gOuMMbunrHMFcCt2gnoj8O/GmDcWsu1MtmzZYnbu3HlcJzRr8hE33kQ/ydoV5DwBHMlxPJGDIHJYW9voDgadS7F8ddRlh/CMHyBrYMzZTLv3bFyWg0ZnjKQV5HdrP0j3aJy2Oh9bN7XokPIS6ugNc+dj+wn5XAS8TsYTGX71Uj9vDQ1weuTXNGb6CSW6GaCBPquVDaEMfjPO0tEXyOayRPwryblqiRsPJjZMY26Q7tAWMt56nKMHaI3uojewiXRw5VE/C7WxTpoDXnocLQylPSw3fayM7+ZgzUY6aaHJleQU5xAI7E03MZT20ORKssafOe6EOL1toberhBh0u8qNYabt1mw673j+iQMgIs8YY7Yc0V5Agjof+EdjzO/lX98OYIz53JR1/gt41BhzT/71HuBiYNVs287keBPUvvYnGXroC+BwknXWYKUjeKM9iEOIe1vJuWppHnkGby5Gn38jWW89YrI0Dz+LkGMktBFnLokrG2VJ9GUSuBnzLGNEgvQnLNazHxdZXml6O43OJN7MOD8PvpuP3vhHRceqCtfRG54cONFW52NoPIHLaRH0OokkM+xvf5qr0z9hjFocvhA1JsYZsSd5kbXU13qpMXE8JkFd/ABekgzXrEEQfPFeLJMm6fARDpxKThzUhTswCMP1Z2LEgUFYEn6RdMbQV3c2TstB09guJB3BuP0MBE8nnTWsGHsOI9AVtP94SWcN3vEuWnPd9Pg3kXX5cUe6WJZ4hV7vqST9bYd/Pn2tZF1+rHQUb7QbcTjybbUzbHfkOqXcrhJi0O2q53cjgCM1RtNlHznuJHW0BFXIJb42YOpTkV3YvaTZ1mkrcNuS6X/qR4g7RE1kP1Y2CUBTxp5qfcjkIA6h3AhZhGWR3zGcbAXAb8YAw3i0n5S4GBMfhnpqSPCKZxOWy0VM0nSlDY1mmGWOYcZdrTxV9w6sxvXzdToqb/rAiYleFdh1DocDa/lR9A+4NvA8TZkBDjlbeDpxJmK5GXLVk8tBzhjOopdBQgyYZThNmlNMDwm8eHMJ0vFxHOTwmhgOckTGX//Y+jKj+DDEo/sACKb7SeHEk4oSD3sA8GeHAUPD+CuT29Wne/GSpDbWPfnaSZrWxCsMZyPAtM9n3utt2QK2y5Z8u0qIQbebebtKiGH6dmMt55BD6H/qR3PqRc2kkAQlM7RN73YdbZ1CtrV3IHITcBPASSedVEBYR7LGu8jUthJznGzPJwTUDo2AMYzX2YnEGk7hzKWwyBCu34QRC+dgnBzCSPMWRMAhQm+fg1W5AyyrzZK0fASJEhm1uMt/MytPOZfxRIZwPM1NWsJowU0fjr5xWZD+sU38tuHcycuAfdGd/Kn1U5y+FEnLjycbgaibYWs5vsY2jIHRgT68mQiHrAbGG88EwN0fAQyHGt+AYMDkcAzEcAqM1p8OGMxQCr+JMSI+xuo2AOAaiiAGxurWT37og0ND9Jm6yc9e7dAo49TjIc543Wn5tonP52mT5ze9rXZohHHq8JBYkO0qIQbdrpp+N0LOE8A53kWpFZKguoCpE/IsB3oKXMddwLYAGGPuBO4E+xJfAXEdIRtYjiM+ivHVTbYlpAYcYLz2X+CjwdNYOrqThKMWXD6s5DgxdyOI4EqN2fcdEmO4PS5+5r2RjaaLpmQ/Q64l/Kb5UgJLNtAbTtBW5+Oac5br/aYymalXNfUy4Dm/dwmPPuvnnMRv7N+ftYTH627g7OQTWIkwOU+AuHsJdelBRnwn43QYHMlxEu56EMGdiebvQcUYddTj97rwWVmSlp9hVysNqQ46XSfj9NjJb8RqAhE8TplMiGnxMOBqw5P/7EWddXgyEaLOeozX/oy+/vmc6TNbl9+ufkG3q4QYdLvq+t04EqNkA6UfHFZIgtoBrBWR1UA3cC3w3mnrbANuEZF7sS/hhY0xvSIyWMC2JdPyxncz9NAXyMHkDe54/j8ciY+S8wQQy82IZznJmhac0V6ygeWELv4AYF8idOZv+rVe/AEaGtcf9p/ee3UARMWanrAA1jRfwvb20w8bwOI5dNHk7znTcDJ9G64k07d78vc+02chfcYn+Pm+kclk11m7kjuyW7jIP8jyfPL7VdMHEBHeap48IiFOfPZi7mZC6QFGvGswueyMn8+Z2hZ6u0qIQberrt+NIxWmKf9vp5RmHSQBk6P0/g17qPg3jTGfFZGbAYwxX8sPM/8KsBV7mPn7jTE7j7btbMeb11F8JRhxohaf6T20U1tqebk/eljyAw5bx06IHRU32qqQ7SohBt2ucmOomFF85TCXBKWUUqq6HC1BabFYpZRSFUkTlFJKqYqkCUoppVRF0gSllFKqImmCUkopVZEqchRf/vmpA3PcTRMwVIJwykFjX3jVGjdo7OWisZfOSmNM8/TGikxQpSAiO2catlgNNPaFV61xg8ZeLhr7/NNLfEoppSqSJiillFIV6UROUHeWO4A50NgXXrXGDRp7uWjs8+yEvQellFKqup3IPSillFJVTBOUUkqpinTCJSgR2Soie0Rkr4h8vNzxHIuIfFNEBkSkfUpbg4j8QkReyX+vL2eMRyMiK0TkERHpEJFdIvLhfHvFxy8iXhF5WkReyMf+6Xx7xccOICKWiDwnIj/Nv66WuF8Tkd+JyPMiMjEdT7XEXiciPxKRl/Kf+fOrIXYRWZd/vye+xkTkr6shdjjBEpSIWMAdwOXABuA6EdlQ3qiO6VvYc2hN9XHgl8aYtcAv868rUQb4iDFmPXAe8Ff597oa4k8CbzPGnAmcBWwVkfOojtgBPgx0THldLXEDvNUYc9aUZ3CqJfZ/B7YbY04DzsR+/ys+dmPMnvz7fRbwBuz5+n5CFcQOgDHmhPkCzgcenPL6duD2csc1S8yrgPYpr/cArfmfW4E95Y6xwPP4H+DSaosfqAGexZ4JuuJjB5Zj/4fyNuCn1fSZAV4Dmqa1VXzsQBDYT35QWTXFPi3ey4DfVlPsJ1QPCmgDOqe87sq3VZMWY0wvQP77kjLHMysRWQWcDTxFlcSfv0z2PDAA/MIYUy2x/xvwt0BuSls1xA1ggIdE5BkRuSnfVg2xrwEGgbvzl1a/ISK1VEfsU10L3JP/uSpiP9ESlMzQpuPo55GI+IH/B/y1MWas3PEUyhiTNfZlj+XAuSKyqdwxzUZE3gkMGGOeKXcsx+lCY8xm7EvwfyUiby53QAVyApuBrxpjzgaiVOolsaMQETfwLuCH5Y6lGCdaguoCVkx5vRzoKVMsx6tfRFoB8t8HyhzPUYmICzs5fc8Y8+N8c9XED2CMGQUexb4XWOmxXwi8S0ReA+4F3iYi36Xy4wbAGNOT/z6AfR/kXKoj9i6gK9/LBvgRdsKqhtgnXA48a4zpz7+uithPtAS1A1grIqvzfzFcC2wrc0zF2ga8L//z+7Dv7VQcERHgLqDDGPPFKYsqPn4RaRaRuvzPPuAS4CUqPHZjzO3GmOXGmFXYn+1fGWOup8LjBhCRWhEJTPyMfT+knSqI3RjTB3SKyLp809uB3VRB7FNcx+uX96BaYi/3TbB5uBF4BfAy8CrwiXLHM0us9wC9QBr7r7Q/Axqxb4K/kv/eUO44jxL7RdiXT18Ens9/XVEN8QNnAM/lY28H/iHfXvGxTzmHi3l9kETFx419H+eF/NeuiX+b1RB7Ps6zgJ35z8x9QH0VxV4DHAJCU9qqInYtdaSUUqoinWiX+JRSSp0gNEEppZSqSJqglFJKVSRNUEoppSqSJiillFIVSROUUkqpiqQJSimlVEXSBKWUUqoiaYJSSilVkTRBKaWUqkiaoJRSSlUkTVBKKaUqkiYopZRSFUkTlFJVSES+JSKfKXccSs0nTVBqURORW0Rkp4gkReRb05Y1iMhPRCQqIgdE5L3FLC8nETlbRH4rIjEReVpETirBPgs+32O9r0oVShOUWux6gM8A35xh2R1ACmgB/hj4qohsLGJ5WYjIcuAB4F+xJ6bbB3yyBLsu5nyP9b4qVRBNUKoqiMjNIvIzEblDRIZEpEdELp3rfo0xPzbG3Ic94+jU49UCfwT8vTEmYoz5DfY02TcUsvwo5/BxEXlVRMZFZLeI/MGUZa+JyEdF5EURCYvI90XEO2X52SLybH7b7wPeGQ9i+wLwdWPMNmNMHLgXOKe4d+aI2Is636O9r0oVQxOUqhZnAOdj/6e4BPgv4GNTVxCRn4rI6FG+flrk8U4FssaYl6e0vQBsLHD5TF4F3gSEgE8D3xWR1inLrwa2Aqvz5/sn+fNyY08z/h2gAfghdrI4gogEgauAb0xpdgCJGdYt5v06nvNVak6c5Q5AqQKdAfyLMeZBABHZjf2f/SRjzDtLeDw/EJ7WFgYCBS4/gjHmh1Nefl9EbgfOBf4n3/YfxpgeABG5Hzgr334e4AL+zRhjgB+JyG1HOczb8+u+KCITbZ4px5gaTzHvV9Hnq9RcaQ9KVYvTgfunvN4E7J7H40WA4LS2IDBe4PIjiMiNIvL8RC8F+xyapqzSN+XnGHZSAFgGdOeT04QDRznMKmCbMaZu4gt4BNh+tLgKVPT5KjVXmqBUxROR1di9/T1Tms8Gnp+23s9FJHKUr58XediXAaeIrJ3Sdiawq8Dl089hJfB14BagMZ842gGZaf1peoE2mdIlAo42Ks+Dndwmjrsa2IJ9aXR6TMW8X0Wdr1KloAlKVYMzgN8ZY3JT2s7GvgcyyRhzuTHGf5Svy2fasYg484MRLMASEa+IOI0xUeDHwP8RkVoRuRD73s538sc65vIZ1AIGGMwf9/3YPahCPAFkgA/l4/1D7EuDM9kBvEVElonICuD/Ap8wxgxPX7GY96vY8z3a+1rg+SoFaIJS1eEMpvSWRKQRWIrdA5mrTwJx4OPA9fmfJ4ZkfxDwAQPAPcBfGmOm9hhmWz7JGLMbe3TdE0A/9iXL3xYSoDEmBfwh9qCJEeAa7GQxk19hXwp9GfgN8B1jzNcLOU4Bjnm++R7Z3+VfHut9VaogcvhlbaWUUqoyaA9KKaVURdIEpZRSqiJpglJKKVWRNEEppZSqSBU57LOpqcmsWrWq3GEopZRaAM8888yQMaZ5entFJqhVq1axc+fOcoehlFJqAYjIjJVR9BKfUkqpiqQJSimlVEXSBKWUUqoiaYJSSilVkSpykIRSSs2XXC5HV1cX0Wi03KEsGi6XiyVLlhAMTp+x5dg0QSmlFpWhoSFEhHXr1uFw6EWk+WaMIR6P093dDVBUktIENY86esNsb++nezROW52PrZtaWN8aKndYSi1qo6OjrFq1SpPTAhERampqaGtro6enp6gEpb+hedLRG+bOx/YTjqdpDXkJx9Pc+dh+Onqnz5qtlFpI2WwWl8tV7jAWHZ/PRzqdLmob7UGV0NQe08HhGC0BD5FkBgFCPvsfxPb2fu1FKVVmh09OrBbC8bzn2oMqkek9poGxBE/tO8RLvWP0hOMABLxOukfjZY5UKaWqgyaoEtne3k/I5yKYT0LpbA4EoskMsWQWgPFEhrY6X5kjVUpVqlWrVvHwww8f0f7rX/+adevWLXg8xRz3W9/6FhdddFFJj68JqkS6R+MEvE6Goym6R+KsbqqlrsZNKpsjmckyHE0SjqfZuqml3KEqparMm970Jvbs2bNojjtB70GVSFudj3A8TSyVRYDNJ9Xz2lCUA8MxIsksToeDm958kt5/UqpK6ajchac9qBLZuqmFcDzNSCyJyxLGEhksy8E/XLmBSze0cNXZbfphVqpKLeSo3B07drBhwwbq6+t5//vfTyKR4NFHH2X58uWT66xatYrPf/7znHHGGYRCIa655hoSicTk8q9//euccsopNDQ08K53vYuenp7JZSLCf/7nf7J27VoCgQB///d/z6uvvsr5559PMBjk6quvJpVKARxx3H/5l3/h5JNPJhAIsGHDBn7yk5+U/Pyn0h5UiaxvDXHTm1fz+QdfJpFJE/K5uOac5Zy2NMjT+0cYGk+WO0Sl1DSP7hlgsIB/mw/t6iOeyhJJvD5MOp7K8sWHXuayjUuPuW1zwMPF65YUHNP3vvc9HnzwQWpra7nyyiv5zGc+wyWXXHLEej/4wQ/Yvn07Xq+XCy+8kG9961vcfPPN/OpXv+L222/noYceYuPGjXz0ox/l2muv5bHHHpvcdvv27TzzzDN0dnayefNmHn/8cb73ve/R2NjI+eefzz333MP73ve+I4558skn8+tf/5qlS5fywx/+kOuvv569e/fS2tpa8PkVQ3tQJbS+NcQ5qxu4+S0n8zeXnsr61hAiQpPfzVBEE5RS1WoklsbrOvy/S6/LwUisuOd6CnHLLbewYsUKGhoa+MQnPsE999wz43of+tCHWLZsGQ0NDVx55ZU8//zzgJ3g/vRP/5TNmzfj8Xj43Oc+xxNPPMFrr702ue3HPvYxgsEgGzduZNOmTVx22WWsWbOGUCjE5ZdfznPPPTfjMd/znvewbNkyHA4H11xzDWvXruXpp58u+XswQXtQJZRIZ4mnspPPPE1o8nvY3TuGMUafv1CqghTas+kaiROOpw/7tx2Op9noc/GeLStKGtOKFa/vb+XKlYddnptq6dLXe241NTWT6/X09LB58+bJZX6/n8bGRrq7u5mYqbyl5fXBWj6f74jXfX19Mx7z29/+Nl/84hcnk10kEmFoaKi4EyyCJqgSih58gfMO/l/WDoVh/xpoXg+DHZzZsw9vIkRk9fUEVp1V7jCVUkXauqmFOx/bD9jPM44nMoTjaa45Z/ksWxavs7Nz8ueDBw+ybNmyorZftmwZBw68PkFtNBrl0KFDtLW1zSmuAwcO8Od//uf88pe/5Pzzz8eyLM466yyMMXPa77HoJb5S6WvH+dRX8GTGcNYvh6G98PCn4NCruBra8GTGME98Gfrayx2pUqpIE/eYQz4XveEEIZ+Lm968el4GPt1xxx10dXUxPDzMP//zP3PNNdcUtf173/te7r77bp5//nmSySR/93d/xxvf+MbJ3tPxikajiAjNzc0A3H333bS3z+//Z9qDKpWO+4k7AqQdgjfWB0MvAQKDHfhql5B0BolKimDH/bB0U7mjVUoVaX1raEFG4r73ve/lsssuo6enh6uuuopPfvKTRd3nefvb384//dM/8Ud/9EeMjIxwwQUXcO+99845rg0bNvCRj3yE888/H4fDwY033siFF1445/0ei8xn9+x4bdmyxezcubPcYRTnvg+yLxkiOdrDevcgDL0CDhfk0nDSeTyXaKXW7eBU3xj8/n+WO1qlFq2Ojg7Wr19f7jAWpaO99yLyjDFmy/R2vcRXKqEVZBNj1JAEhxOa1oKv3v5KRfC5LbLxMIRKe0NVKaVOVJqgSmX9lRAfpSYzCk4f+FshFYFAK6QThHJjmMQomXXvLHekSilVFTRBlUh2yUaebLkOl0MgE4emU+CST0PzqZAcx+P1sLP1eob9a8sdqlJKVQUdJFEiY/E0YW8bqbZzYfOVsDx/OXXtJfD4l7Ha3syhA80MRVIsCXrLG6xSSlWBgnpQIrJVRPaIyF4R+fgMy08TkSdEJCkiH53SvkJEHhGRDhHZJSIfLmXwlSQcT1OTHsHrsqC26fUFHj+4awlkhnE6RCtKKKVUgWbtQYmIBdwBXAp0ATtEZJsxZveU1YaBDwG/P23zDPARY8yzIhIAnhGRX0zb9oQQjqfxZUbw1lhQ23z4Qn8L/b0HePZgC79+ZZBnDoxoJWSllJpFIT2oc4G9xph9xpgUcC9w1dQVjDEDxpgdQHpae68x5tn8z+NABzC3x5kr1Gg8TSAzgstbC+7aw5YdSNXyXMeriMnithzzWglZKaVOFIUkqDagc8rrLo4jyYjIKuBs4Klit60G4XiaBhlDpveegP/tdlDjhFW+OOmcodZtEfK52N7eX4ZIlVKqOhSSoGaqblrU070i4gf+H/DXxpixo6xzk4jsFJGdg4ODxey+IoSjSepM+MjLe8Cr8VrcTgeNjAKQzOQI5KeGV0qpudi4cSOPPvpoucOYF4UkqC5g6tOly4GZy+vOQERc2Mnpe8aYHx9tPWPMncaYLcaYLRO1nqqFMYZ4ZJQaRxZqG49YXtfYQjzroC43AkAqm2M8kaGtzrfQoSqlKtiqVavw+Xz4/X5aWlp4//vfTyQSOeY2u3bt4uKLL16YABdYIQlqB7BWRFaLiBu4FthWyM7FnlviLqDDGPPF4w+zssVSWVyJQ/kRfEcm162ntzJkgrjjQxhjGImmCMfTbN3UMsPelFIVqa8dHvkc3PdB+/s8FX6+//77iUQiPPvss+zYsYPPfOYz83KcajBrgjLGZIBbgAexBzn8wBizS0RuFpGbAURkqYh0AbcBnxSRLhEJAhcCNwBv5QTQ8QAAIABJREFUE5Hn819XzNvZlMloPI0vPYLH6ZgxQa1vDXHBmetpcYSJJDK4nY55q4SslJoHfe3w+JchPgrBNvv74/M7O0FbWxuXX3457e3tbNu2jY0bN1JXV8fFF19MR0fH5HqrVq3i4YcfBuDpp59my5YtBINBWlpauO222wBIJBJcf/31NDY2UldXxznnnEN/v30PvKenh3e96100NDRwyimn8PWvf31y3//4j//I1VdfzY033kggEGDjxo0sZJ3Ugh7UNcY8ADwwre1rU37uw770N91vmPke1gklHLOfgfI0BME182W7FSetZkXiFV7NBVnR2qrJSalK8MrDEClgsNJLP4VUDJJjMDH4NhWDRz4Lp81SvszfYj+wX6TOzk4eeOABzjjjDK677jruu+8+Lr74Yr70pS9x5ZVXsnv3btxu92HbfPjDH+bDH/4wN9xwA5FIZHI6jP/+7/8mHA7T2dmJx+Ph+eefx+ez/6+67rrr2LhxIz09Pbz00ktceumlrFmzhre//e0AbNu2jR//+MfcfffdfPKTn+SWW27hySefLPp8joeWOpqjjt4wd/1mH68dPMD/dpujDx3325fzGs0o0VRmASNUSs1ZbPjIPz5dPru9xH7/93+furo6LrroIt7ylrewYcMG3vGOd3DppZficrn46Ec/Sjwe5/HHHz9iW5fLxd69exkaGsLv93PeeedNth86dIi9e/diWRZveMMbCAaDdHZ28pvf/IZ//dd/xev1ctZZZ/GBD3yA73znO5P7vOiii7jiiiuwLIsbbriBF154oeTnfDRa6mgOOnrD3PnYfkajCd7oHOeQWcNDj+2f+fJd/tJfvQnTmcyWIVql1BEK7dmMHrQv6/nqXm+Lj0LrmXD2H5c0pPvuu49LLnk9rr/8y79k5cqVk68dDgcrVqygu7v7iG3vuusu/uEf/oHTTjuN1atX86lPfYp3vvOd3HDDDXR2dnLttdcyOjrK9ddfz2c/+1l6enpoaGggEAhM7mPlypWHXcabPrV8IpEgk8ngdM5/+tAe1Bxsb+8n5HMRIIbXkUP8TUd/vmnoFejayYaXv8K6Dp1ZV6mqsv5KSIzaScnk7O+JUbt9nk2fwt0YQ2dn54xTuK9du5Z77rmHgYEBPvaxj/Hud7+baDSKy+XiU5/6FLt37+bxxx/npz/9Kd/+9rdZtmwZw8PDjI+PT+7j4MGDc54evlQ0Qc1B92icgNeJJzWM0+Eg5qqf+fmmiRusgMPy4EiGyc3zDValVAkt3QQX3Gr3oMa67e8X3Logs2NfffXV/OxnP+OXv/wl6XSaL3zhC3g8Hi644IIj1v3ud7/L4OAgDoeDujq7t2dZFo888gi/+93vyGazBINBXC4XlmWxYsUKLrjgAm6//XYSiQQvvvgid911F3/8x6XtFR4vvcQ3B211PjyHdnNF+F7W5vbR0OfgWf9baWucNmNkx/3grQNx4Iq+SsqqJeN24tbp35WqHks3leXf67p16/jud7/LrbfeSnd3N2eddRb333//EQMkALZv385tt91GLBZj5cqV3HvvvXi9Xvr6+rj55pvp6urC7/dzzTXXcP311wNwzz33cPPNN7Ns2TLq6+v59Kc/zaWXXrrQpzkjnfJ9Dva1P8nQQ1+A+DBBF4zWrMaRCtN02UdYs+m811e874P20NTYMPHudnbJyZy2ogV/sl+nf1dqgemU7+WjU74voDWDj3DSslYscRDDg/HVcfJJbawZfOTwFUMrIDEGTg+WQ3DmUmR0+nellDomTVBzEe6kNtRAk8+wtq2Z89c00tjQDOHOw9ebuMGaieMQ8KZHMLGRBbnBqpRS1UoT1FyEVpCNjeIwWSyXx25LjB3ZM5q4wVrTjCMdIedwc3DdB/T+k1JKHYMOkpiL9VfCQ/+KlUtiWc7Xh55uvuHIdfM3WB2+Orq6nThqT1n4eJVSgD1U2y4VqhbK8Yx30B7UXCzdRN/J7yHr8OBMhQsbeuoJ4Ceh1SSUKhPLskin07OvqEoqHo/jcrmK2kZ7UHM07mkhXPcGlr/jNggum30DTwA/3exPaoJSqhzq6uro7++nra0Nh0P/Rp9vxhji8Tjd3d20tBQ3g4MmqDnKJMbxWA7E7S9sA0+AGuJEE/oXnFLl0NTURFdXF3v27Cl3KIuGy+WipaWFYDBY1HaaoOYoE49Qawm4awvbwBvC48iRSkTJ5QwOh14HV2ohORwOTjrppHKHoQqg/ds5yiUjONw14LAK28ATwG05cGeieh9KKaWOQRPUHGUTEcRT4OU9AE8Al9OBOxMhltKq5kopdTSaoOYqFcHyBGZfb4InaPegslEiOlBCKaWOShPUHKSzORzpKJa3iB6UuxaXy4knGyGqCUoppY5KE9QcxFMZXNk4Ll8RPSgRXL6g9qCUUmoWmqDmIB6P4zAZnMX0oACHN0hA4kR1Zl2llDoqTVBzkIyOAeCuKW5sP94gfmJ6iU8ppY5BE9QcJGP2NMmemiIu8QF4AtSaOBF9WFcppY5KE9QcpOL5BFVbZA/KE8TjMKQSkXmISimlTgyaoOYgHR9HOJ4eVBCX00E2Fiabq7wZjZVSqhJogpqDbGIcy+lEXDXFbZivJuHKRIlpNQmllJpRQQlKRLaKyB4R2SsiH59h+Wki8oSIJEXko8VsW80yiXHEXQvFziszUU0iG9WRfEopdRSzJigRsYA7gMuBDcB1IrJh2mrDwIeAzx/HtlXLJCOIp8AisVO5a3G7XHiyEX0WSimljqKQHtS5wF5jzD5jTAq4F7hq6grGmAFjzA5g+rC0WbetZrlkFKuYOnwTRBjJeHitu4/PPdDBl37xMh294dIHqJRSVayQBNUGdE553ZVvK0TB24rITSKyU0R2Dg4OFrj78jKpCA5vkQMkgI7eML/pTOLKRAh6nYTjae58bL8mKaWUmqKQBDXTDZZCh54VvK0x5k5jzBZjzJbm5uYCd18+2WwWScWKriIBsL29H4c3SL2VJJ0zhHwuQj4X29v75yFSpZSqToUkqC5gxZTXy4GeAvc/l20rWjwWQTA4i6nDl9c9GsfhC1FLjEzGHiQR8DrpHo2XOkyllKpahSSoHcBaEVktIm7gWmBbgfufy7YVLRG1L8d5jiNBtdX5GM16cYnBkbGT0ngiQ1udr6QxKqVUNZs1QRljMsAtwINAB/ADY8wuEblZRG4GEJGlItIF3AZ8UkS6RCR4tG3n62QWUipqV5FwFfuQLrB1UwtDaTdZk8NKjxOOpwnH02zd1FLqMJVSqmo5C1nJGPMA8MC0tq9N+bkP+/JdQdueCCbKHHmLLRQLrG8NUXOak9Qjv6M59ioO50Za3vge1rSGSh2mUkpVrYISlDpSKmZXMvf6i09Q9LWz8uCPidRCQprZsNTCevluaPLD0k0ljlQppaqTljo6TpnEODmHC6/nOO4bddwPNY2IqwaLNGlXELx1drtSSilAE9RxyyYiGLcfh6PIMkcA4U7whhCXByuXJp3LgTdotyullAI0QR23bDJi1+E7HqEVkBjD4fRgmTTprIHEmN2ulFIK0AR13HKJCHI8ZY4A1l8JiVEsk8HKpsjFRiAxarcrpZQCNEEdv1Tk+OrwgT0Q4oJbsWrq8WVGSDiDcMGtOkBCKaWm0FF8xyObIZdO4PQdZ4ICWLoJx7l/xsHIT4if9hcsX1poeUOllFocNEEVq6+d3Is/4KSBR8k5x6Bv1fH3fNy1uCxhNDZe0hCVUupEoJf4itHXDo9/mWxkiKRVi8uk4PEv2+3Hw+3HZTlIxzVBKaXUdJqgitFxP4dyPl7oSzASS9MRdnEo5zv+55fyCSobHyttnEopdQLQBFWEkd597OjNkk4lsRxC0ljs6M0y0rvv+HborsXlEDKJSGkDVUqpE4AmqCK8FA9RJ3E8kgPA5fZQJ3Feih9nDT13LU6nE5McJ5crdIotpZRaHDRBFeG3rvMJEMWdGSOH4MtFCRDlt67zj2+HIlieWpyZOIn8vFBKKaVsmqCKYLWezv82XUtaXHhJkHYF+d+ma7FaTz/ufTprArizMWIpTVBKKTWVDjMvwtZNLdz5WAy35zyMROhech3heJqb5jCPk9MbwJ3tJK4JSimlDqM9qCKsbw1x05tXUyMpxrNuQj4XN715NevnMI+TyxfAldMelFJKTac9qCKtbw2RaXGzzGrm4ktPnfP+PDUhXNkEsWSqBNEppdSJQ3tQx8GkYliempLsy10TQDAktZqEUkodRhNUsbIZTCZZsgQlngAuS0hpglJKqcNogipWJk4mm8M63rmgpnPX4rQcpLWahFJKHUYTVJFMKkY6a3B6S5egXJaDbEJ7UEopNZUmqCKlElEM4PKUKkH5cVlCRgvGKqXUYTRBFSkVjwLgmstcUFNZThxuH7mk1uNTSqmpNEEVKZ0v7OouVYICLG8QSUVJZ3Ml26dSSlW7ghKUiGwVkT0isldEPj7DchGR/8gvf1FENk9Z9jcisktE2kXkHhHxlvIEFloqbicoj680o/gAnF6/ljtSSqlpZk1QImIBdwCXAxuA60Rkw7TVLgfW5r9uAr6a37YN+BCwxRizCbCAa0sWfRlkElEyDg9et6tk+3Tmq0louSOllHpdIT2oc4G9xph9xpgUcC9w1bR1rgK+bWxPAnUi0ppf5gR8IuIEaoCeEsVeFplklIzDi89llWyf7pqQ3YNKpku2T6WUqnaFJKg2oHPK665826zrGGO6gc8DB4FeIGyMeWimg4jITSKyU0R2Dg4OFhr/gssmY6QtL96SJqgAYnLE47GS7VMppapdIQlKZmibPrvejOuISD1272o1sAyoFZHrZzqIMeZOY8wWY8yW5ubmAsIqj2wqBi4flmOmUz4+npogACl9WFcppSYVkqC6gBVTXi/nyMt0R1vnEmC/MWbQGJMGfgxccPzhll8uaSeoUnL5AlgOIRXTBKWUUhMKSVA7gLUislpE3NiDHLZNW2cbcGN+NN952JfyerEv7Z0nIjUiIsDbgY4Sxr+wjMGkY1ilekh3gtuPyyGktR6fUkpNmnW6DWNMRkRuAR7EHoX3TWPMLhG5Ob/8a8ADwBXAXiAGvD+/7CkR+RHwLJABngPunI8TWRDZNNlMGstduiHmAHj8uCwHGb3Ep5RSkwqaD8oY8wB2Epra9rUpPxvgr46y7aeAT80hxsqRjpHJmtL3oCw3lstFNqHVJJRSaoJWkihGOk4ml8NZoqk2JnT0jbFrKMfTL3fypV+8TEdvuKT7V0qpaqQJqggm34NylaqSOdDRG+bOx/YTNV5CkiAcT3HnY/s1SSmlFj2d8r0IqbhdydzpLV0dvu3t/aw1BzgrtQNfvJ+BgTFe9L+Z7e01rG8Nlew4SilVbbQHVYR0wq5k7vaVrgeV7f0dbxm6F5fJksSNOzXGW4buJdv7u5IdQymlqpEmqCLYhWIFj7d096AuTD/BOLUkXUEcGOKOGsap5cL0EyU7hlJKVSNNUEVIJ6KkHV587tJdGT3NF2bU+Egae5+ZZIJR4+M0n96DUkotbpqgipBJRslYnpLW4atvXcM5rRYuj5dszuCWLOe0WtS3rinZMZRSqhppgipCNhkj7fDidZXwbVt/JY2OOJtbvTTUuFgXTNHoiMP6K0t3DKWUqkKaoIowMdWG11m6HhRLN8EFtyL+ZrwmShIXXHCr3a6UUouYDjMvQi4VA/dSHCWsZA7YyahlI919SSJNZ7FCk5NSSmkPqmDGYFIxHKWuwzdBBIc3QDah9fiUUgo0QRUukySbzZa+UOwUDk+AXEIrmiulFGiCKlw6RjprsEpch28qpy+ASUawa+8qpdTipgmqUJOFYktcyXwKyxfAlY0ST2fn7RhKKVUtNEEVKh0nkzU4S1godjp3TQgrlyYWi83bMZRSqlpogipQLhUlm5vfBOWpCQKQiGgVCaWU0gRVoHTCrmReykKx03lr7erliagmKKWU0gRVoFQ8ihEHHrdv3o7h8dsJKhXTBKWUUpqgCpRO5gvFeubv2WZPTRCHQCqmQ82VUkorSRSirx13+/dZO9RJYKcTzvrDeSlFJC4vlstDJq4P6yqllPagZtPXDo9/GRJjxFz1uFJj9uu+9nk5nHj8ZPVhXaWU0gQ1q477wVtHVixyDhdWTT146+z2eeDwBjVBKaUUmqBmF+4EbxCTTZMTJ06HgDdot88DyxvAaIJSSilNULMKrYD4KCabAcuNiEBizG6fB5YvgKQi5LK5edm/UkpVi4ISlIhsFZE9IrJXRD4+w3IRkf/IL39RRDZPWVYnIj8SkZdEpENEzi/lCcy79VcSHu5jJDzGwZEkz768n9GRwXmbUNDtCyImSzwenZf9K6VUtZg1QYmIBdwBXA5sAK4TkQ3TVrscWJv/ugn46pRl/w5sN8acBpwJdJQg7gXTYVbwndTFJHDRaEUJm1ruTL+DDjM/PShPrV1NIhYdnZf9K6VUtShkmPm5wF5jzD4AEbkXuArYPWWdq4BvG7sM95P5XlMrEAXeDPwJgDEmBaRKF/78297ejyO4glfGzuLJ0OUsWbGWWDzN9vZ+1reGSn48T76aRDIyBi0l371SSlWNQi7xtQFTRwR05dsKWWcNMAjcLSLPicg3RGTGWkEicpOI7BSRnYODgwWfwHzrHo1T50yTM4as064iEfA66R6Nz8vxJsodJbXckVJqkSskQc00v/n0CYuOto4T2Ax81RhzNnaP6oh7WADGmDuNMVuMMVuam5sLCGthtNX5SCci5Azk8glqPJGhrW5+Sh75/HUApPRhXaXUIldIguoCpt5wWQ70FLhOF9BljHkq3/4j7IRVNbZuaiEZGyeZNRjLSzieJhxPs3XT/Fx/c3s84PSQ0XJHSqlFrpAEtQNYKyKrRcQNXAtsm7bONuDG/Gi+84CwMabXGNMHdIrIuvx6b+fwe1cVb31riMtODZK1fIzGM4R8Lm568+p5uf80yePXckdKqUVv1kESxpiMiNwCPAhYwDeNMbtE5Ob88q8BDwBXAHuBGPD+Kbu4FfhePrntm7asKjR7sixrbuT2res5ZYl/3o/n8AS0moRSatErqFisMeYB7CQ0te1rU342wF8dZdvngS1ziLHsMokIGcuLz20tyPEc3gBm9OCCHEsppSqVVpIoQCYRJe3wUeNamATl9AUwySiY6WNRlFJq8dAEVYBMMkp6AXtQLl+IbDZDNqnVJJRSi5cmqNlkM+RSCbLOGjzOhXm73DV2NYl4RKtJKKUWL01Qs0nHSGdzWO4au1DsAphIUImojuRTSi1eOqPubPIJyuGd/9F7E/yJXpaHn8H7yGuw8g12Ydp5mMFXKaUqmfagZpOOkckZXJ4ZKzSVXl877LybeCzKCwPY1dN/+cV5m8FXKaUqlSao2aTsHpTLtzA9qIEdP2TXiIMYPnzOLGOmlheG7HallFpMNEHNJh0jnTW4FypBde7FeILkLDdWLo3HZWHcAQY69y7I8ZVSqlJogppFNhEhkwOPt2ZBjtdtmghInKzDjZVNAhCQON2maUGOr5RSlUIT1CySiQhpy0eNZ2HGk/QvuwwrFcZJDkc2iSc9hpUK07/ssgU5vlJKVQpNULNIxyNkHD58C1RF4pzzLuKh4NWMOQLU5MYZNx4eCl7NOeddtCDHV0qpSqHDzGeRSUQWtIrE+tYQ/N4lbHvYoqt7O+MrruHKN5w+v9XTlVKqAmmCmsVEHb6F6kGBnaTkvA2MPPYbNp1ZT0CTk1JqEdJLfLOYqMNX417YXF4TqAcgPn5oQY+rlFKVQhPUsWQz5FJJslYNXtfCvlW1tQGy4iIxPrKgx1VKqUqhCepY0lG7zJHHt2B1+CYEfG5SzlrSUS0Yq5RanDRBHUsqRjpncHoWrg7fBLfTQdYTJB3THpRSanHSBHUs6RiZbA7Lu0B1+KaxfPVkY+GyHFsppcpNE9Sx5CuZu32BshzeVRsim4xBJlmW4yulVDlpgjqW1EQdvvL0oFw19aSyOUjovFBKqcVHE9QxZJNR0kbweBamDt90nkAD6azR+1BKqUVJE9QxpBPj9kO6C1SHbzqf334WKjY2XJbjK6VUOWmCOoZUPGoXil2gMkfT1QZCGHGQ1GehlFKLkCaoY8gkImQc3gUtczRVwOciaflJRLQHpZRafApKUCKyVUT2iMheEfn4DMtFRP4jv/xFEdk8bbklIs+JyE9LFfhCyCSiC1oodjq/x0nSGSCjD+sqpRahWROUiFjAHcDlwAbgOhHZMG21y4G1+a+bgK9OW/5hoGPO0S6wbDJC2lG+S3xOywHeEBl9FkoptQgV0oM6F9hrjNlnjEkB9wJXTVvnKuDbxvYkUCcirQAishx4B/CNEsY9/zIpMukUGcuH11meBAXg9IXIJschmylbDEopVQ6FJKg2oHPK6658W6Hr/Bvwt0DuWAcRkZtEZKeI7BwcHCwgrHnU1w6PfJbQwYfYOPRzHAO7yhaKy19PKpODpD4LpZRaXApJUDNVSTWFrCMi7wQGjDHPzHYQY8ydxpgtxpgtzc3NBYQ1T/ra4fEvQ/QQaYcfDyn7dV97WcJx19aRzOQgofehlFKLSyEJqgtYMeX1cqCnwHUuBN4lIq9hXxp8m4h897ijXQgd94O3Dlw+skDWW2+/7ri/LOH4Ag1kc4ZkRIeaK6UWl0IS1A5grYisFhE3cC2wbdo624Ab86P5zgPCxpheY8ztxpjlxphV+e1+ZYy5vpQnUHLhTvAGIZciZwyW022/DnfOvu088AXqMQhxfRZKKbXIzFoiwRiTEZFbgAcBC/imMWaXiNycX/414AHgCmAvEAPeP38hz7PQCoiPQiZFLmewXB67Fl5oxezbzoOAz02vVas9KKXUolNQDR9jzAPYSWhq29em/GyAv5plH48CjxYd4UJbfyU8/mVMfIS0sfBko5BIwuYbyhKO3+sk6fST0od1lVKLjFaSmG7pJrjgVrLixGkSiK8OLrjVbi8D//BLrBp5grrd34VHPle2wRpKKbXQNEHNoMOs4OHker6T28pXzHvoMOW5vEdfO44nv4JLcqTwQGy4rCMKlVJqIWmCmqajN8ydj+0nFw9j3H4S6Sx3Prafjt4yVHPIjyg0vgZyxoDbV9YRhUoptZA0QU2zvb2fBo/BKxniDj8NtW5CPhfb2/sXPphwJ4MZN6+NGfrHEjz3ai+DGXfZRhQqpdRC0gQ1TfdonEZXkmzOEJMaXJaDgNdJ92h8wWMZsJrZvb+bBG4cIphUlN37uxmwyvggs1JKLRBNUNO01flIx8L2w7GOGlyWMJ7I0FbnW/BYtmfOJSRRglaKDBbB7CghibI9c+6Cx6KUUgtNE9Q0Wze1kImGiaezZFwBwvEM4XiarZtaFjyWFzPLeWH5DaRdIVykyQIvLL+BFzPLFzwWpZRaaOWZy7yCrW8NUXN6kOd/7WA44+ZUn4trzlnO+tbQgsfSVudjf3w1PStvQlIrOcvTzX5rFW0B94LHopRSC017UDNYWZujpamBD7zlVP7m0lPLkpzA7s2F42kS6SzjjhCpRJxEdKwsvTmllFpomqBmkI2PMm5qCHjL28H8/9u79+C47uqA49+z79WutKuXZUWSn5hYQYEGHOdByARIiAmYUAZKKBAKpSYDoYGBYYDO9PEHpX/QTIHhUTdAYWDCQICQMNQJ0LQkhTh2bGJkK04cO1iSJVlWtKvVal939/SPXQdFlvHaXueutOczo7H297tXe/aMdI9/9/e79/Z3x9h27VriTQFO0IKI8MFXRV0rmMYY82KyU3yLyKWT5L0R1wsUlItUf3eMHe0eugZ38pJI3u2QjDHmRWEjqEXk00nyvggtIb/boTyvpSVGuuijmDrudijGGPOisAK1UCFLPpch543WxQjqpPZoiDl/K+mEFShjTGOwArVQLkXeKZH3RogG66dAtUb8ZHxxckkX7mhhjDEusAK1UG6GnFPC29SCz1s/6WltCpANtJKbm4V82u1wjDHmgqufI3C9yM2Qd0oEI61uR/ICfq8HiXaSKRRhbsrtcIwx5oKzArVQLkWuqISiLW5HcoqmeBeZfBHSJ9wOxRhjLjgrUAtodoZZDdIcDrkdyimaY22kix7UCpQxpgFYgVogP5cgIxFa6mgF30ltkSBpb4xM0lbyGWOWPytQC+Rmy9dANdfRNVAntUb8zPlbySRsJZ8xZvmzAjWfKoW5JDlvfY6g2iNBMv5WcukkFLJuh2OMMRdU/R2F3eRkyeey5L3RuhxBhQNemjVFaOQ38JPboPNi6N8KKwfcDs0YY2rORlDz5VLknBKlQJSQvw5TMz7IwIkHKRWyEGqGTAJ+82UYH3Q7MmOMqbk6PAq7qHIXCX8kjoi4Hc2phu5Ho53kJFg+xReOQygOQ/e7HZkxxtRcVQVKRLaIyEEROSQin16kX0TkS5X+fSLyykp7n4g8JCJDIrJfRO6o9QeoqWySnFMiGIm7HcniksP4I3FyEqKYTZXbQi2QHHY3LmOMuQDOOAclIl7gK8ANwAiwS0TuU9UD8zZ7I7Ch8nUF8LXKvw7wCVXdIyLNwOMi8osF+9aPXIpcsUQ4WqfPW4r14YweYyTjpzAyxnTmIjbES7S397kdmTHG1Fw1I6jNwCFVPayqeeD7wM0LtrkZ+I6WPQrERaRbVcdUdQ+AqqaAIaCnhvHXzvggpd3fomfyYTYe/Gpdzusc7nwtYxPjOCXwUSIwN84zR0c53Plat0Mzxpiaq6ZA9QDzzyGNcGqROeM2IrIGuAzYudibiMg2EdktIrsnJyerCKuGxgfhN1/GST/HnK+NcDFVl4sPfjrWxu6L3k3Sv4JgKU3A52H3Re/hp2NtbodmjDE1V80y88VWC+jZbCMiUeBHwMdUdWaxN1HV7cB2gE2bNi38+RfW0P0QilNMTFD0+vBH2oB0ub2OlnCPJjJ0t/fzs1wPmZkO4vGVZNo2MpbIuB32uictAAAO3UlEQVSaMcbUXDUjqBFg/iRHL3Cs2m1ExE+5OH1PVX987qFeQMlhCEQoFTIUPCGCPk9dLj7oiYdJZR2aQz5G6SCaHSeVKdATD7sdmjHG1Fw1BWoXsEFE1opIALgFuG/BNvcBt1ZW810JJFV1TMprtb8BDKnqnTWNvJZifTA7QbFYouBtIuDzQHam3F5Htgx0kcwUEIHj0kl2Lk1xdpItA11uh2aMMTV3xgKlqg5wO/AA5UUOP1DV/SJym4jcVtns58Bh4BDwH8CHK+2vBt4LvE5Eflf5uqnWH+K89W8lNTXKVGKGQ9Mlfvf0sySmJ8t3aagj/d0xtl27loviYf7gtKPABwa89HfX6apDY4w5D1Xd6khVf065CM1v+/q87xX4yCL7PcLi81N1ZUj72J+7lI1M0+VLkdRetheuYav20e92cAv0d8fo747x3ZYQ3cN7WOufdjskY4y5IOxefMCOwQnWh5r4bfR69rbdxIauZuYyBXYMTtTt6KSnrYnRo+0MJIbtdiDGmGXJjm3Asek0cZ1mUuM0BbwA5YUIdbw6rjceJuHvIj3zXPmefMYYs8xYgQLWRQtkMjkS3lailcdspLJOXa+O62kNMxNcSSrjQHLE7XCMMabmrEABb1gNc3mHiVIL4YCXZKZAMlOo69VxTQEfK70z+EYfhQc+Cw99vu4uLDbGmPNhBQpYH56jpy2CJ9LBZCpPLOxn27Vr63b+CYDxQS4fu5uMI6jHb4/eMMYsO7ZIAtDUBLlAG+98+Vq2DHS7HU51hu4n1NzG9JxDITtFIND0fHs93f3CGGPOlRUoIJcYY9rTyqpY/c45nSI5TNbTyjPpIN6ZNIXCM6zsXU1nnd39whhjzpWd4sunSacSzPnbWdkScjuaqh33dvL08BhZCZH3hAlkJzlwZJTj3k63QzPGmJqwAjU7wWzOIRvqoCMacDuaqu1wNhOTNJ2+DFPEaS4laNdpdjib3Q7NGGNqwgrU7HFmsw5Nbd34vEsnHfucXp7ofS/FYAwPBXKEONZ6OfucXrdDM8aYmmj4OahSapzpUpjO1la3QzkrPfEwRzJrmVx/G3uOJri++Ai9+Wl6uoNuh2aMMTWxdIYMtTY+CA99HufhL7Fiei+rCkfcjuisnLyz+WyuSHvEz55cD8Vsirf01e/dL4wx5mw0ZoGqPEGXuSnylUFk79BdS+oaopN3No+F/SjCZLCP9c0l1u+7E+79sF24a4xZ8hrzFF/lCbp4vBScErlQB4Fo65K7hujknc0BHvjlL4nsP4jmS8i61/zxwt2rP7qkPpMxxpzUmCOo5DCTToDBw0cZTWQYSXs54QTr7gm6Z+OyzCNM+bvJ4YP0FITj5SI8dL/boRljzDlpyAJ13NvJgcMjBDLHyXgiOBJY8tcQtTsTTHtaeSoBB586wM5nJph0Aku66BpjGltDFqgdzmY6dYqm4gzTnjY6fBlikl7S1xBN+VYwl0pwuLSSkKdIOD2y5IuuMaaxNWSB2uf0MhF7BbMaJigFisEYT/S+d0lfQ7TD2UyXP4Pfo0zSRkfhGB2lqSVddI0xja0hF0msbgZnMs1PQm9jvOsa1rRHSGYK9DT73Q7tnO1zetFVt9Ix8iDpzCwO0MI0G5/9Djy0D/q32mIJY8yS0pAjqK1dJ0jOZRnyrGNlS3BJPP/pTHriYY741jJ08Yd5sOUdHC+1QDFPxO+1R3EYY5akxipQ44Pw0D/T+9jnuESepT+aYSpdWBrPfzqD+Rfu3uB5jGFt5+lsDE9qhL1Hn2OqFLYVfcaYJaVxTvGND5L41Z2MJLJ4EjmmPM38je9ndLzmk7DypW5Hd95OXri7Y3ACX2qUmVILWX8vK30jxGefZjgVpenYTsLJYYj12Sk/Y0zda5gCdXzXDxmaVNpLszjiIxHqYf9zWfp3/ZAVW5fHgfrkhbu/PbqKizMJpkpNDOXWcGnxSdZnn+BEpoUh8dI7vIe1v7+XcOc6WPlyK1bGmLq0rAvU4cFHmdh5D97UCJ2zB+n0teLDYczbSWs0hOP4OT58iBVuB1pj/+e/ijdm7sEf8DChAUrJNHPqJU2Ai/P7CWYnGXU8FGeHSc8085LDf0/LitVQzNnoyhhTN6oqUCKyBfgi4AXuUtV/WdAvlf6bgDngr1R1TzX71trJohQ8MUgsN4Yvsp5icw+BxB7iziRP+i4hE+sh5vEQlllGSx0st0Oxt/tS/jfg47L0w6wqjeGXIg/zKoKibE4P4ikVCEkQKRZI5dKkJp8iMXmMJ6OX03l4N12P/pAp/0pmWy/Bv/ISCuMH8KZGKEoAEcVTKlBs7rU+67M+66PY3EvXFW9n3cCVNT+Wiar+6Q1EvMBTwA3ACLALeJeqHpi3zU3ARykXqCuAL6rqFdXsu5hNmzbp7t27z/rDHB58lBMP/iulQIyWxH6aijP4cJjzt1PIZ4mRIuWJk7joGoLFNJ5ckn19t3Lrn7/prN+rng2NJdn+6yPEwn6aQz46dt9Ji85COM7LUo9QVIgzC4DP46FQKiHiYbJpA/G5IzjqwROKccLTSffcEMeiAzi+MN3JvaAw3vpKvPk03en9jDUP4Hitz/qsr1H78Ibw5JN0vOET51ykRORxVd10SnsVBeoq4B9V9cbK688AqOrn523z78D/qOrdldcHgeuANWfadzHnWqB++41PIpkEofQwvZkhCvjwUCJPgP3Rq3GyaS4uPk2242Wc8K5gV+gatt54/ZJevXc6Q2NJdgxOMJrIoOODvLNwL75IK76JQaLMUdISz8hq+ktPIZQQwEMJL0WKeAlSIE8AHzkcgiiChyIARXwoECBHniAKeCkiz/cpAfIUnu9zEMCpDNhP7of1WZ/1Lfm+kd6bkGwKDce56q+/wLk4XYGq5hRfDzD/hm4jlEdJZ9qmp8p9Twa4DdgGsGrVqirCOpU3NYIT6Safj5HItSIKWU+YoGaJtXaQmxH2Fq/msRUfoiceZutA17IsTvDCO50Pja3h/gdCXJ59hIivmYiT5g/e1fja1pI+MU5E0wz6X86GwgHyBAlKnhn10ESGtDYRIgtAliYQCGoOAdJECJIHqPziCgFyAGRoImB91md9DdAnlILN+FIj1Fo1BUoWaVs47DrdNtXsW25U3Q5sh/IIqoq4TlFs7sWTSVCMreKEN8SK5BMES1ly3giSSdCks2za8iH+YuAV5/Ljl6z+7hjceD07Bi9lfziJb/IANwcfp1enOOJbT8yZpKUlQi7ZTLiYRgSOBjfS4wwTdGZJ+srLSLylcmHKeOIABJ1Zpn1tL+hLe1qf70tbn/VZ3/LvEw+ebIJic+1vFVfNhbojQN+8173AsSq3qWbfmum64u148kkkk6DU1M5UeB0iSj7Qgobj53WOdKnr747x8Rteyl3vu5w73v1Wntx4O99e8Ske2fhZHujaRt7XAqEYJVUO0Ycn2sFcoJOQzjEXWsFMZDWhUppgMU0qusb6rM/6rI9UdA2SSeDJJ+m64u01P25VMwflo7zQ4fXAKOWFDn+pqvvnbfMm4Hb+uEjiS6q6uZp9F3Ouc1DwwqXlF3J1yXIyf76qL/8MA8lf05wfr7uVQtZnfdZXn33ne5w95zkoVXVE5HbgAcpLxb+pqvtF5LZK/9eBn1MuTocoLzN//5/a95w/RRXWDVxpBekszZ+vglcAb3MzHGOMAaoYQbnhfEZQxhhjlpbTjaAa62axxhhjlgwrUMYYY+qSFShjjDF1yQqUMcaYumQFyhhjTF2qy1V8IjIJ/OE8f0wHcKIG4Sw3lpfFWV4WZ3lZnOVlceeal9Wq2rmwsS4LVC2IyO7Fli02OsvL4iwvi7O8LM7ysrha58VO8RljjKlLVqCMMcbUpeVcoLa7HUCdsrwszvKyOMvL4iwvi6tpXpbtHJQxxpilbTmPoIwxxixhVqCMMcbUpWVXoERki4gcFJFDIvJpt+Nxi4j0ichDIjIkIvtF5I5Ke5uI/EJEnq782+p2rG4QEa+I7BWRn1VeN3xeRCQuIveIyJOV35urLC8gIh+v/A0NisjdIhJqxLyIyDdF5LiIDM5rO20eROQzlePwQRG58Vzec1kVKBHxAl8B3ghcArxLRC5xNyrXOMAnVLUfuBL4SCUXnwZ+paobgF9VXjeiO4Chea8tL/BFYIeqbqT8YLAhGjwvItID/C2wSVUHKD/X7hYaMy//CWxZ0LZoHirHmluAl1X2+Wrl+HxWllWBAjYDh1T1sKrmge8DN7sckytUdUxV91S+T1E+2PRQzse3K5t9G3irOxG6R0R6gTcBd81rbui8iEgLcC3wDQBVzatqggbPS4UPCFeeEN4EHKMB86KqvwaeW9B8ujzcDHxfVXOqeoTyw2w3n+17LrcC1QMMz3s9UmlraCKyBrgM2Al0qeoYlIsYsMK9yFzzb8CngNK8tkbPyzpgEvhW5dTnXSISocHzoqqjwBeAo8AYkFTVB2nwvMxzujzU5Fi83AqULNLW0OvoRSQK/Aj4mKrOuB2P20TkzcBxVX3c7VjqjA94JfA1Vb0MSNMYp63+pMqcys3AWuAiICIi73E3qiWhJsfi5VagRoC+ea97KQ/HG5KI+CkXp++p6o8rzRMi0l3p7waOuxWfS14NvEVEnqV8Cvh1IvJdLC8jwIiq7qy8vodywWr0vFwPHFHVSVUtAD8GrsbyctLp8lCTY/FyK1C7gA0islZEApQn6e5zOSZXiIhQnk8YUtU753XdB7yv8v37gJ++2LG5SVU/o6q9qrqG8u/Hf6vqe7C8jAPDInJxpen1wAEaPC+UT+1dKSJNlb+p11Oez230vJx0ujzcB9wiIkERWQtsAB472x++7O4kISI3UZ5j8ALfVNXPuRySK0TkGuBh4Pf8ca7ls5TnoX4ArKL8x/cOVV048dkQROQ64JOq+mYRaafB8yIif0Z54UgAOAy8n/J/Yhs9L/8EvJPyyti9wAeBKA2WFxG5G7iO8iM1JoB/AO7lNHkQkb8DPkA5bx9T1f866/dcbgXKGGPM8rDcTvEZY4xZJqxAGWOMqUtWoIwxxtQlK1DGGGPqkhUoY4wxdckKlDHGmLpkBcoYY0xd+n/ew5AhmBPX7gAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/poisson_11_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "def binomial(k, n, p):\n", + " # Binomial(n, p) pmf evaluated at k\n", + " return binom(n, k) * p**k * (1-p)**(n-k)\n", + "\n", + "θ_vals = 0.5, 0.2, 0.1\n", + "\n", + "n_vals = 50, 75, 100\n", + "\n", + "fig, axes = plt.subplots(len(n_vals), 1, figsize=(6, 12))\n", + "\n", + "for n, θ, ax in zip(n_vals, θ_vals, axes.flatten()):\n", + "\n", + " k_grid = np.arange(n)\n", + " binom_vals = [binomial(k, n, θ) for k in k_grid]\n", + " poisson_vals = [poisson(k, n * θ) for k in k_grid]\n", + " ax.plot(k_grid, binom_vals, 'o-', alpha=0.5, label='binomial')\n", + " ax.plot(k_grid, poisson_vals, 'o-', alpha=0.5, label='Poisson')\n", + " ax.set_title(f'$n={n}$ and $\\\\theta = {θ}$')\n", + " ax.legend(fontsize=12)\n", + "\n", + "fig.tight_layout()\n", + "plt.show()" + ] + } + ], + "metadata": { + "jupytext": { + "formats": "ipynb,md:myst", + "text_representation": { + "extension": ".md", + "format_name": "myst", + "format_version": "0.9", + "jupytext_version": "1.5.0" + } + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.7" + }, + "source_map": [ + 13, + 38, + 44, + 72, + 92, + 174, + 196, + 257, + 282, + 452, + 493, + 501 + ] + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file diff --git a/_sources/poisson.md b/_sources/poisson.md new file mode 100644 index 0000000..12bfeff --- /dev/null +++ b/_sources/poisson.md @@ -0,0 +1,526 @@ +--- +jupytext: + formats: ipynb,md:myst + text_representation: + extension: .md + format_name: myst + format_version: '0.9' + jupytext_version: 1.5.0 +kernelspec: + display_name: Python 3 + language: python + name: python3 +--- + + +# Poisson Processes + + +## Overview + +Counting processes count the number of "arrivals" occurring by a given time +(e.g., the number of visitors to a website, the number of customers arriving at a restaurant, etc.) + +Counting processes become Poisson processes when the time interval between +arrivals is IID and exponentially distributed. + +Exponential distributions and Poisson processes have deep connections to +continuous time Markov chains. + +For example, Poisson processes are one of the simplest nontrivial examples of +a continuous time Markov chain. + +In addition, when continuous time Markov chains jump between states, the time +between jumps is *necessarily* exponentially distributed. + +In discussing Poisson processes, we will use the following imports: + +```{code-cell} ipython3 +import numpy as np +import matplotlib.pyplot as plt +import quantecon as qe +from numba import njit +from scipy.special import factorial, binom +``` + + + +## Counting Processes + +Let's start with the general case of an arbitrary counting process. + + +### Jumps and Counts + +Let $(J_k)$ be an increasing sequence of nonnegative random variables +satisfying $J_k \to \infty$ with probability one. + +For example, $J_k$ might be the time the $k$-th customer arrives at a shop. + +Then + +$$ + N_t := \sum_{k \geq 0} k \mathbb{1} \{ J_k \leq t < J_{k+1} \} +$$ (defcount) + +is the number of customers that have visited by time $t$. + + + +The next figure illustrate the definition of $N_t$ for a given jump sequence $\{J_k\}$. + +```{code-cell} ipython3 +:tags: [hide-input] + +Ks = 0, 1, 2, 3 +Js = 0, 0.8, 1.8, 2.1, 3 +n = len(Ks) + +fig, ax = plt.subplots() + +ax.plot(Js[:-1], Ks, 'o') +ax.hlines(Ks, Js[:-1], Js[1:], label='$N_t$') +ax.vlines(Js[:-1], (0, Ks[0], Ks[1], Ks[2]), Ks, alpha=0.25) + +ax.set(xticks=Js[:-1], + xticklabels=[f'$J_{k}$' for k in range(n)], + yticks=(0, 1, 2, 3), + xlabel='$t$') + +ax.legend(loc='lower right') +plt.show() +``` + +An alternative but equivalent definition is + +$$ + N_t = \max \{k \geq 0 \,|\, J_k \leq t \} +$$ + +As a function of $t$, the process $N_t$ is called a **counting process**. + +The jump times $(J_k)$ are sometimes called **arrival times** and the +intervals $J_k - J_{k-1}$ are called **wait times** or **holding times**. + + + + +### Exponential Holding Times + +A Poisson process is a counting process with independent exponential holding times. + +In particular, suppose that the arrival times are given by $J_0 = 0$ and + +$$ + J_k := W_1 + \cdots W_k +$$ + +where $(W_i)$ are IID exponential with some fixed rate $\lambda$. + +Then the associated counting process $(N_t)$ is called a **Poisson process** with rate $\lambda$. + +The rationale behind the name is that, for each $t > 0$, the random variable +$N_t$ has the Poisson distribution with parameter $t \lambda$. + +In other words, + +$$ + \PP\{N_t = k\} + = e^{-t \lambda} \frac{(t \lambda)^k }{k!} + \qquad (k = 0, 1, \ldots) +$$ (poissondist) + +For example, since $N_t = 0$ if and only if $W_1 > t$, we have + +$$ + \PP\{N_t =0\} + = \PP\{W_1 > t\} + = e^{-t \lambda} +$$ + +and the right hand side agrees with {eq}`poissondist` when $k=0$. + +This sets up a proof by induction, which is time consuming but not difficult +--- the details can be found in $\S29$ of {cite}`howard2017elements`. + +Another way to show that $N_t$ is Poisson with rate $\lambda$ is appeal to +{proof:ref}`erlexp`. + +We observe that + +$$ + \PP\{N_t \leq n\} + = \PP\{J_{n+1} > t\} + = 1 - \PP\{J_{n+1} \leq t\} +$$ + +Inserting the expression for the Erlang CDF in {eq}`erlcdf` with shape $n+1$ and +rate $\lambda$, we obtain + +$$ + \PP\{N_t \leq n\} + = \sum_{k=0}^{n} \frac{(t \lambda )^k}{k!} e^{-t \lambda} +$$ + +This is the (integer valued) CDF for the Poisson distribution with parameter +$t \lambda$. + +An exercise at the end of the lecture asks you to verify that $N(t)$ is Poisson-$(t \lambda )$ informally via simulation. + +The next figure shows one realization of a Poisson process $(N_t)$, with jumps +at each new arrival. + + +```{code-cell} ipython3 +:tags: [hide-input] + +np.random.seed(1234) +T = 5 +Ws = np.random.exponential(size=T) +Js = np.cumsum(Ws) +Ys = np.arange(T) + +fig, ax = plt.subplots() + +ax.plot(np.insert(Js, 0, 0)[:-1], Ys, 'o') +ax.hlines(Ys, np.insert(Js, 0, 0)[:-1], Js, label='$N_t$') +ax.vlines(Js[:-1], Ys[:-1], Ys[1:], alpha=0.25) + +ax.set(xticks=[], + yticks=range(Ys.max()+1), + xlabel='time') + +ax.grid(lw=0.2) +ax.legend(loc='lower right') +plt.show() +``` + + + +## Stationary Independent Increments + +One of the defining features of a Poisson process is that it has stationary +and independent increments. + +This is due to the memoryless property of exponentials. + +It means in particular that + +1. the variables $\{N_{t_{i+1}} - N_{t_i}\}_{i \in I}$ are independent for any + strictly increasing finite sequence $(t_i)_{i \in I}$ and +2. the distribution of $N_{t+h} - N_t$ depends on $h$ but not $t$. + + +A detailed proof can be found in Theorem 2.4.3 of {cite}`norris1998markov`. + +Instead of repeating this, we provide some intuition from a discrete +approximation. + +In the discussion below, we use the following well known fact: If +$(\theta_n)$ is a sequence such that $n \theta_n$ converges, then + +$$ + \text{Binomial}(n, \theta_n) + \approx + \text{Poisson}(n \theta_n) + \quad \text{for large } n +$$ (binpois) + +(The exercises ask you to examine this claim visually.) + +We now return to {ref}`the environment ` where we linked the +geometric distribution to the exponential. + +That is, we fix small $h > 0$ and let $t_i := ih$ for all $i \in \ZZ_+$. + +Let $(V_i)$ be IID binary random variables with $\PP\{V_i = 1\} = h \lambda$ for some $\lambda > 0$. + +Linking to our previous discussion, + +* either one or zero customers visits a shop at each $t_i$. +* $V_i = 1$ means that a customer visits at time $t_i$. +* Visits occur with probability $h \lambda$, which is proportional to the + length of the interval between grid points. + +We learned that the wait time until the first visit is +approximately exponential with rate $t \lambda$. + +Since $(V_i)$ is IID, the same is true for the second wait time and so on. + +Moreover, these wait times are independent, since they depend on separate +subsets of $(V_i)$. + +Let $\hat N_t$ count the number of visits by time $t$, as shown in the next figure. + +($V_i = 1$ is indicated by a vertical line at $t_i = i h$.) + +```{code-cell} ipython3 +:tags: [hide-input] + +fig, ax = plt.subplots() +np.random.seed(1) +T = 10 +p = 0.25 +B = np.random.uniform(size=T) < p +N = np.cumsum(B) +m = N[-1] # max of N + +t_grid = np.arange(T) +t_ticks = [f'$t_{i}$' for i in t_grid] +ax.set_yticks(range(m+1)) +ax.set_xticks(t_grid) +ax.set_xticklabels(t_ticks, fontsize=12) + +ax.step(t_grid, np.insert(N, 0, 0)[:-1], label='$\hat N_t$') + +for i in t_grid: + if B[i]: + ax.vlines((i,), (0,), (m,), ls='--', lw=0.5) + +ax.legend(loc='center right') +plt.show() +``` + +We expect from the discussion above that $(\hat N_t)$ approximates a Poisson process. + +This intuition is correct because, fixing $t$, letting $k := \max\{i \in +\ZZ_+ \,:\, t_i \leq t\}$ and applying {eq}`binpois`, we have + +$$ + \hat N_t + = \sum_{i=1}^k V_i + \sim \text{Binomial}(k, h \lambda) + \approx + \text{Poisson}(k h \lambda ) +$$ + +Using the fact that $kh = t_k \approx t$ as $h \to 0$, we see +that $\hat N_t$ is approximately Poisson with rate $t \lambda$, just as we +expected. + + +This approximate construction of a Poisson process helps illustrate the +property of stationary independent increments. + +For example, if we fix $s, t$, then $\hat N_{s + t} - \hat N_s$ is the number of visits +between $s$ and $s+t$, so that + +$$ + \hat N_{s+t} - \hat N_s + = \sum_i V_i \mathbb 1\{ s \leq t_i < s + t \} +$$ + +Suppose there are $k$ grid points between $s$ and $s+t$, so that $t \approx +kh$. + +Then + +$$ + \hat N_{s+t} - \hat N_s + \sim \text{Binomial}(k, h \lambda ) + \approx + \text{Poisson}(k h \lambda ) + \approx + \text{Poisson}(t\lambda) +$$ + +This illustrates the idea that, for a Poisson process $(N_t)$, we have + +$$ + N_{s+t} - N_s + \sim \text{Poisson}(t\lambda) +$$ + +In particular, increments are stationary (the distribution depends on $t$ but not $s$). + +The approximation also illustrates independence of increments, since, in the +approximation, increments depend on separate subsets of $(V_i)$. + + + + +## Uniqueness + +What other counting processes have stationary independent increments? + +Remarkably, the answer is none: + +```{proof:theorem} Characterization of Poisson Processes + +If $(M_t)$ is a stochastic process supported on $\ZZ_+$ and starting at 0 with +the property that its increments are stationary and independent, then $(M_t)$ is a Poisson process. + +``` + +In particular, there exists a $\lambda > 0$ such that + +$$ + M_{s + t} - M_s + \sim \text{Poisson}(t\lambda) +$$ + +for any $s, t$. + +The proof is similar to our earlier proof that the exponential distribution is +the only memoryless distribution. + +Details can be found in Section 6.2 of {cite}`pardoux2008markov` or +Theorem 2.4.3 of {cite}`norris1998markov`. + +(restart_prop)= +### The Restarting Property + +An important consequence of stationary independent increments is the +restarting property, which means that, when simulating, we can freely stop and +restart a Poisson process at any time: + +```{proof:theorem} Poisson Processes can be Pause and Restarted +If $(N_t)$ is a Poisson process, $s > 0$ and +$(M_t)$ is defined by $M_t = N_{s+t} - N_s$ for $t \geq 0$, then $(M_t)$ is a +Poisson process independent of $(N_r)_{r \leq s}$. +``` + +```{proof:proof} +Independence of $(M_t)$ and $(N_r)_{r \leq s}$ follows from indepenence of the +increments of $(N_t)$. + +In view of the uniqueness statement above, we can verify that $(M_t)$ is a +Poisson process by showing that $(M_t)$ starts at zero, takes values in +$\ZZ_+$ and has stationary independent increments. + +It is clear that $(M_t)$ starts at zero and takes values in $\ZZ_+$. + +In addition, if we take any $t < t'$, then + +$$ + M_{t'} - M_t = N_{s+t'} - N_{s + t} + \sim \text{Poisson}((t' - t) \lambda) +$$ + +Hence $(M_t)$ has stationary increments and, +using the relation $M_{t'} - M_t = N_{s+t'} - N_{s + t}$ again, +the increments are independent as well. + +We conclude that $(N_{s+t} - N_s)_{t \geq 0}$ is indeed a +Poisson process independent of $(N_r)_{r \leq s}$. +``` + + + +## Exercises + +Exercise 1 +---------- + +Fix $\lambda > 0$ and draw $\{W_i\}$ as IID exponentials with rate $\lambda$. + +Set $J_n := W_1 + \cdots W_n$ with $J_0 = 0$ and + $N_t := \sum_{n \geq 0} n \mathbb 1\{ J_n \leq t < J_{n+1} \}$. + +Provide a visual test of the claim that $N_t$ is Poisson with parameter $t +\lambda$. + +Do this by fixing $t = T$, generating many independent draws of $N_T$ and +comparing the empirical distribution of the sample with a Poisson +distribution with rate $T \lambda$. + +Try first with $\lambda = 0.5$ and $T=10$. + +Exercise 2 +---------- + + +In the lecture we used the fact that $\Binomial(n, \theta) \approx \Poisson(n \theta)$ when $n$ is large and $\theta$ is small. + +Investigate this relationship by plotting the distributions side by side. + +Experiment with different values of $n$ and $\theta$. + + +## Solutions + + +### Solution to Exercise 1 + +Here is one solution. + +The figure shows that the fit is already good with a modest sample size. + +Increasing the sample size will further improve the fit. + + +```{code-cell} ipython3 +λ = 0.5 +T = 10 + +def poisson(k, r): + "Poisson pmf with rate r." + return np.exp(-r) * (r**k) / factorial(k) + +@njit +def draw_Nt(max_iter=1e5): + J = 0 + n = 0 + while n < max_iter: + W = np.random.exponential(scale=1/λ) + J += W + if J > T: + return n + n += 1 + +@njit +def draw_Nt_sample(num_draws): + draws = np.empty(num_draws) + for i in range(num_draws): + draws[i] = draw_Nt() + return draws + + +sample_size = 10_000 +sample = draw_Nt_sample(sample_size) +max_val = sample.max() +vals = np.arange(0, max_val+1) + +fig, ax = plt.subplots() + +ax.plot(vals, [poisson(v, T * λ) for v in vals], + marker='o', label='poisson') +ax.plot(vals, [np.mean(sample==v) for v in vals], + marker='o', label='empirical') + +ax.legend(fontsize=12) +plt.show() +``` + + +### Solution to Exercise 2 + +Here is one solution. It shows that the approximation is good when $n$ is +large and $\theta$ is small. + +```{code-cell} ipython3 +def binomial(k, n, p): + # Binomial(n, p) pmf evaluated at k + return binom(n, k) * p**k * (1-p)**(n-k) + +θ_vals = 0.5, 0.2, 0.1 + +n_vals = 50, 75, 100 + +fig, axes = plt.subplots(len(n_vals), 1, figsize=(6, 12)) + +for n, θ, ax in zip(n_vals, θ_vals, axes.flatten()): + + k_grid = np.arange(n) + binom_vals = [binomial(k, n, θ) for k in k_grid] + poisson_vals = [poisson(k, n * θ) for k in k_grid] + ax.plot(k_grid, binom_vals, 'o-', alpha=0.5, label='binomial') + ax.plot(k_grid, poisson_vals, 'o-', alpha=0.5, label='Poisson') + ax.set_title(f'$n={n}$ and $\\theta = {θ}$') + ax.legend(fontsize=12) + +fig.tight_layout() +plt.show() +``` + + diff --git a/_sources/prob_view.ipynb b/_sources/prob_view.ipynb new file mode 100644 index 0000000..df7cc32 --- /dev/null +++ b/_sources/prob_view.ipynb @@ -0,0 +1,270 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# A Probabilistic View\n", + "\n", + "\n", + "## Overview\n", + "\n", + "Let $S$ be a countable state space and let $Q$ be a conservative intensity\n", + "matrix on $S$.\n", + "\n", + "We know that $Q$ generates a UC Markov semigroup, and from this semigroup we\n", + "can produce a continuous time Markov chain via the associated joint distribution.\n", + "\n", + "This method of building chains from intensity matrices is important from a\n", + "theoretical perspective but less helpful in terms of intuition and simulation.\n", + "\n", + "In this lecture we provide another point of view.\n", + "\n", + "In particular, we provide two different ways to build a Markov chain with\n", + "intensity matrix $Q$.\n", + "\n", + "One is via jump chains with state dependent jump rates.\n", + "\n", + "The second is another probabilistic construction, sometimes called Gillespie's\n", + "Algorithm.\n", + "\n", + "These two constructions provide intuition from multiple perspectives and\n", + "valuable simulation algorithms.\n", + "\n", + "We will use the following imports" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import scipy as sp\n", + "import matplotlib.pyplot as plt\n", + "import quantecon as qe\n", + "from numba import njit\n", + "from scipy.linalg import expm" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## From Intensity Matrix to Jump Chain\n", + "\n", + "Let us agree to call $(\\lambda, K)$ a **jump chain pair** if $\\lambda$ is a\n", + "map from $S$ to $\\RR_+$ and $K$ is a Markov matrix on $S$.\n", + "\n", + "It is easy to verify that the matrix $Q$ on $S$ defined by \n", + "\n", + "$$\n", + " Q(x, y) := \\lambda(x) (K(x, y) - I(x, y))\n", + "$$ (jcinmat)\n", + "\n", + "is an intensity matrix.\n", + "\n", + "(We saw in {doc}`an earlier lecture ` that $Q$ is the intensity\n", + "matrix for the jump chain $(X_t)$ built via {proof:ref}`ejc_algo` from jump\n", + "chain pair $(\\lambda, K)$.)\n", + "\n", + "As we now show, every intensity matrix admits the decomposition in\n", + "{eq}`jcinmat` for some jump chain pair.\n", + "\n", + "\n", + "(constlk)=\n", + "### Construction \n", + "\n", + "Given a intensity matrix $Q$, set \n", + "\n", + "$$\n", + " \\lambda(x) := -Q(x, x)\n", + " \\qquad (x \\in S)\n", + "$$\n", + "\n", + "Next we build $K$, first along the principle diagonal via\n", + "\n", + "$$\n", + " K(x,x) = \n", + " \\begin{cases}\n", + " 0 & \\text{ if } \\lambda(x) > 0\n", + " \\\\\n", + " 1 & \\text{ otherwise}\n", + " \\end{cases}\n", + "$$\n", + "\n", + "Thus, if the rate of leaving $x$ is positive, we set $K(x,x) = 0$, so that the\n", + "embedded jump chain moves away from $x$ with probability one when the next\n", + "jump occurs.\n", + "\n", + "Otherwise, when $Q(x,x) = 0$, we stay at $x$ forever, so $x$ is an **absorbing\n", + "state**.\n", + "\n", + "Off the principle diagonal, where $x \\not= y$, we set\n", + "\n", + "$$\n", + " K(x,y) = \n", + " \\begin{cases}\n", + " \\frac{Q(x,y)}{\\lambda(x)} & \\text{ if } \\lambda(x) > 0\n", + " \\\\\n", + " 0 & \\text{ otherwise }\n", + " \\end{cases}\n", + "$$\n", + "\n", + "The exercises below ask you to confirm that, for $\\lambda$ and $K$ just defined, \n", + "\n", + "1. $(\\lambda, K)$ is a jump chain pair and\n", + "1. the intensity matrix $Q$ satisfies {eq}`jcinmat`.\n", + "\n", + "We call $(\\lambda, K)$ the **jump chain decomposition** of $Q$.\n", + "\n", + "We summarize in a lemma.\n", + "\n", + "```{proof:lemma}\n", + ":label: imatjc\n", + "\n", + "A matrix $Q$ on $S$ is an intensity matrix if and only if there exists a jump\n", + "chain pair $(\\lambda, K)$ such that {eq}`jcinmat` holds.\n", + "```\n", + "\n", + "\n", + "### The Conservative Case\n", + "\n", + "\n", + "We know from {proof:ref}`jccs` that an intensity matrix $Q$ is conservative\n", + "if and only if $\\lambda$ is bounded.\n", + "\n", + "Moreover, we saw in {proof:ref}`usmg` that the pairing between conservative\n", + "intensity matrices and UC Markov semigroups is one-to-one.\n", + "\n", + "This leads to the following result.\n", + "\n", + "```{proof:theorem}\n", + "On $S$, there exists a one-to-one correspondence between the following sets of\n", + "objects:\n", + "\n", + "1. The set of all jump chain pairs $(\\lambda, K)$ such that $\\lambda$ is bounded.\n", + "1. The set of all conservative intensity matrices.\n", + "1. The set of all UC Markov semigroups.\n", + "\n", + "```\n", + "\n", + "\n", + "\n", + "### Simulation\n", + "\n", + "\n", + "In view of the preceding discussion, we have a simple way to simulate a Markov\n", + "chain given any conservative intensity matrix $Q$.\n", + "\n", + "The steps are\n", + "\n", + "1. Decompose $Q$ into a jump chain pair $(\\lambda, K)$.\n", + "2. Simulate via {proof:ref}`ejc_algo`.\n", + "\n", + "\n", + "Recalling our discussion of the Kolmogorov backward equation, we know that \n", + "this produces a Markov chain with Markov semigroup\n", + "$(P_t)$ where $P_t = e^{tQ}$ for $Q$ satisfying {eq}`jcinmat`.\n", + "\n", + "(Although our argument assumed finite $S$, the proof goes through when\n", + "$S$ is contably infinite and $Q$ is conservative with very minor changes.)\n", + "\n", + "In particular, $(X_t)$ is a continuous time Markov chain with intensity matrix\n", + "$Q$.\n", + "\n", + "\n", + "\n", + "## The Gillespie Algorithm\n", + "\n", + "Exponential clocks.\n", + "\n", + "Show by simulation that distributions coincide with $\\psi_0 P_t$.\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "## Exercises\n", + "\n", + "### Exercise 1\n", + "\n", + "Let $Q$ be any intensity matrix on $S$.\n", + "\n", + "Prove that the jump chain decomposition of $Q$ is in fact a jump chain pair.\n", + "\n", + "Prove that, in addition, this decomposition $(\\lambda, K)$ satisfies {eq}`jcinmat`.\n", + "\n", + "## Solutions\n", + "\n", + "### Solution to Exercise 1\n", + "\n", + "Let $Q$ be an intensity matrix and let $(\\lambda, K)$ be the jump chain\n", + "decomposition of $Q$.\n", + "\n", + "Nonnegativity of $\\lambda$ is immediate from the definition of an intensity\n", + "matrix.\n", + "\n", + "To see that $K$ is a Markov matrix we fix $x \\in S$ and suppose first that \n", + "$\\lambda(x) > 0$.\n", + "\n", + "Then \n", + "\n", + "$$\n", + " \\sum_y K(x, y) \n", + " = \\sum_{y \\not= x} K(x,y) \n", + " = \\sum_{y \\not= x} \\frac{Q(x,y)}{\\lambda(x)}\n", + " = 1\n", + "$$\n", + "\n", + "If, on the other hand, $\\lambda(x) = 0$, then \n", + "$\\sum_y K(x, y) = 1$, is immediate from the definition.\n", + "\n", + "As $K$ is nonnegative, we see that $K$ is a Markov matrix.\n", + "\n", + "Thus, $(\\lambda, K)$ is a valid jump chain pair.\n", + "\n", + "The proof that $Q$ and $(\\lambda, K)$ satisfy {eq}`jcinmat` is mechanical and\n", + "the details are omitted.\n", + "\n", + "(Try working case-by-case, with $\\lambda(x) = 0, x=y$, $\\lambda(x) > 0, x=y$, etc.)" + ] + } + ], + "metadata": { + "jupytext": { + "formats": "ipynb,md:myst", + "text_representation": { + "extension": ".md", + "format_name": "myst", + "format_version": "0.9", + "jupytext_version": "1.5.0" + } + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.7" + }, + "source_map": [ + 13, + 45, + 52 + ] + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file diff --git a/_sources/prob_view.md b/_sources/prob_view.md new file mode 100644 index 0000000..1d014d4 --- /dev/null +++ b/_sources/prob_view.md @@ -0,0 +1,231 @@ +--- +jupytext: + formats: ipynb,md:myst + text_representation: + extension: .md + format_name: myst + format_version: '0.9' + jupytext_version: 1.5.0 +kernelspec: + display_name: Python 3 + language: python + name: python3 +--- + + +# A Probabilistic View + + +## Overview + +Let $S$ be a countable state space and let $Q$ be a conservative intensity +matrix on $S$. + +We know that $Q$ generates a UC Markov semigroup, and from this semigroup we +can produce a continuous time Markov chain via the associated joint distribution. + +This method of building chains from intensity matrices is important from a +theoretical perspective but less helpful in terms of intuition and simulation. + +In this lecture we provide another point of view. + +In particular, we provide two different ways to build a Markov chain with +intensity matrix $Q$. + +One is via jump chains with state dependent jump rates. + +The second is another probabilistic construction, sometimes called Gillespie's +Algorithm. + +These two constructions provide intuition from multiple perspectives and +valuable simulation algorithms. + +We will use the following imports + +```{code-cell} ipython3 +import numpy as np +import scipy as sp +import matplotlib.pyplot as plt +import quantecon as qe +from numba import njit +from scipy.linalg import expm +``` + + +## From Intensity Matrix to Jump Chain + +Let us agree to call $(\lambda, K)$ a **jump chain pair** if $\lambda$ is a +map from $S$ to $\RR_+$ and $K$ is a Markov matrix on $S$. + +It is easy to verify that the matrix $Q$ on $S$ defined by + +$$ + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +$$ (jcinmat) + +is an intensity matrix. + +(We saw in {doc}`an earlier lecture ` that $Q$ is the intensity +matrix for the jump chain $(X_t)$ built via {proof:ref}`ejc_algo` from jump +chain pair $(\lambda, K)$.) + +As we now show, every intensity matrix admits the decomposition in +{eq}`jcinmat` for some jump chain pair. + + +(constlk)= +### Construction + +Given a intensity matrix $Q$, set + +$$ + \lambda(x) := -Q(x, x) + \qquad (x \in S) +$$ + +Next we build $K$, first along the principle diagonal via + +$$ + K(x,x) = + \begin{cases} + 0 & \text{ if } \lambda(x) > 0 + \\ + 1 & \text{ otherwise} + \end{cases} +$$ + +Thus, if the rate of leaving $x$ is positive, we set $K(x,x) = 0$, so that the +embedded jump chain moves away from $x$ with probability one when the next +jump occurs. + +Otherwise, when $Q(x,x) = 0$, we stay at $x$ forever, so $x$ is an **absorbing +state**. + +Off the principle diagonal, where $x \not= y$, we set + +$$ + K(x,y) = + \begin{cases} + \frac{Q(x,y)}{\lambda(x)} & \text{ if } \lambda(x) > 0 + \\ + 0 & \text{ otherwise } + \end{cases} +$$ + +The exercises below ask you to confirm that, for $\lambda$ and $K$ just defined, + +1. $(\lambda, K)$ is a jump chain pair and +1. the intensity matrix $Q$ satisfies {eq}`jcinmat`. + +We call $(\lambda, K)$ the **jump chain decomposition** of $Q$. + +We summarize in a lemma. + +```{proof:lemma} +:label: imatjc + +A matrix $Q$ on $S$ is an intensity matrix if and only if there exists a jump +chain pair $(\lambda, K)$ such that {eq}`jcinmat` holds. +``` + + +### The Conservative Case + + +We know from {proof:ref}`jccs` that an intensity matrix $Q$ is conservative +if and only if $\lambda$ is bounded. + +Moreover, we saw in {proof:ref}`usmg` that the pairing between conservative +intensity matrices and UC Markov semigroups is one-to-one. + +This leads to the following result. + +```{proof:theorem} +On $S$, there exists a one-to-one correspondence between the following sets of +objects: + +1. The set of all jump chain pairs $(\lambda, K)$ such that $\lambda$ is bounded. +1. The set of all conservative intensity matrices. +1. The set of all UC Markov semigroups. + +``` + + + +### Simulation + + +In view of the preceding discussion, we have a simple way to simulate a Markov +chain given any conservative intensity matrix $Q$. + +The steps are + +1. Decompose $Q$ into a jump chain pair $(\lambda, K)$. +2. Simulate via {proof:ref}`ejc_algo`. + + +Recalling our discussion of the Kolmogorov backward equation, we know that +this produces a Markov chain with Markov semigroup +$(P_t)$ where $P_t = e^{tQ}$ for $Q$ satisfying {eq}`jcinmat`. + +(Although our argument assumed finite $S$, the proof goes through when +$S$ is contably infinite and $Q$ is conservative with very minor changes.) + +In particular, $(X_t)$ is a continuous time Markov chain with intensity matrix +$Q$. + + + +## The Gillespie Algorithm + +Exponential clocks. + +Show by simulation that distributions coincide with $\psi_0 P_t$. + + + + + +## Exercises + +### Exercise 1 + +Let $Q$ be any intensity matrix on $S$. + +Prove that the jump chain decomposition of $Q$ is in fact a jump chain pair. + +Prove that, in addition, this decomposition $(\lambda, K)$ satisfies {eq}`jcinmat`. + +## Solutions + +### Solution to Exercise 1 + +Let $Q$ be an intensity matrix and let $(\lambda, K)$ be the jump chain +decomposition of $Q$. + +Nonnegativity of $\lambda$ is immediate from the definition of an intensity +matrix. + +To see that $K$ is a Markov matrix we fix $x \in S$ and suppose first that +$\lambda(x) > 0$. + +Then + +$$ + \sum_y K(x, y) + = \sum_{y \not= x} K(x,y) + = \sum_{y \not= x} \frac{Q(x,y)}{\lambda(x)} + = 1 +$$ + +If, on the other hand, $\lambda(x) = 0$, then +$\sum_y K(x, y) = 1$, is immediate from the definition. + +As $K$ is nonnegative, we see that $K$ is a Markov matrix. + +Thus, $(\lambda, K)$ is a valid jump chain pair. + +The proof that $Q$ and $(\lambda, K)$ satisfy {eq}`jcinmat` is mechanical and +the details are omitted. + +(Try working case-by-case, with $\lambda(x) = 0, x=y$, $\lambda(x) > 0, x=y$, etc.) diff --git a/_sources/uc_mc_semigroups.ipynb b/_sources/uc_mc_semigroups.ipynb new file mode 100644 index 0000000..6078ebb --- /dev/null +++ b/_sources/uc_mc_semigroups.ipynb @@ -0,0 +1,519 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# UC Markov Semigroups\n", + "\n", + "## Overview\n", + "\n", + "In our previous lecture we covered some of the general theory of continuous\n", + "operator semigroups. \n", + "\n", + "Next we translate these results into the setting of Markov semigroups.\n", + "\n", + "The main aim is to give an exact one-to-one correspondence between UC\n", + "Markov semigroups and \"conservative\" intensity matrices on a countable set $S$.\n", + "\n", + "Conservativeness is defined below and relates to \"nonexplosiveness\" of the\n", + "associated Markov matrix.\n", + "\n", + "We will also give a brief discussion of intensity matricies that fall\n", + "outside this class, along with the processes they generate.\n", + "\n", + "\n", + "## Notation and Terminology\n", + "\n", + "Let $S$ be an arbitrary countable set.\n", + "\n", + "Let $\\ell_1$ be the Banach space of **summable functions** on $S$; that is, all $g \\, \\colon S \\to \\RR$ with \n", + "\n", + "$$\n", + " \\| g \\| := \\sum_x |g(x)| < \\infty\n", + "$$\n", + "\n", + "Note that $\\dD$, the set of distributions on $S$, is contained in $\\ell_1$.\n", + "\n", + "Each Markov matrix $P$ on $S$ can and will be identifed with a linear operator\n", + "$f \\mapsto fP$ on $\\ell_1$ via \n", + "\n", + "$$\n", + " (fP)(y) = \\sum_x f(x) P(x, y)\n", + " \\qquad (f \\in \\ell_1, \\; y \\in S)\n", + "$$ (mmismo)\n", + "\n", + "To be consistent with earlier notation, we are writing the argument of $P$ to\n", + "the left and applying $P$ to it as if premultiplying $P$ by a row vector.\n", + "\n", + "In the exercises you are asked to verify that {eq}`mmismo` defines\n", + "a bounded linear operator on $\\ell_1$ such that \n", + "\n", + "$$\n", + " \\|P\\| = 1 \\text{ and } \\phi P \\in \\dD \\text{ whenever } \\phi \\in \\dD\n", + "$$ (propp)\n", + "\n", + "Note that composition of $P$ with itself is equivalent to powers of the matrix\n", + "under matrix multiplication.\n", + "\n", + "For an intensity matrix $Q$ on $S$ we can try to introduce the associated\n", + "operator analogously, via\n", + "\n", + "$$\n", + " (fQ)(y) = \\sum_x f(x) Q(x, y)\n", + " \\qquad (f \\in \\ell_1, \\; y \\in S)\n", + "$$ (imislo)\n", + "\n", + "However, the sum in {eq}`imislo` is not always well defined.[^fnim]\n", + "\n", + "[^fnim]: Previously, we introduced the notion of an intensity matrix when $S$\n", + " is finite and the definition is essentially unchanged in the current\n", + " setting. In particular, $Q \\colon S \\times S \\to \\RR$ is called an\n", + " **intensity matrix** if $Q$ has zero row sums and $Q(x, y) \\geq 0$ whenever\n", + " $x \\not= y$.\n", + "\n", + "We say that an intensity matrix $Q$ is **conservative** if the sum in\n", + "{eq}`imislo` is well defined at all $y$ and, in addition, the mapping $f\n", + "\\mapsto fQ$ in {eq}`imislo` is a bounded linear operator on $\\ell_1$.\n", + "\n", + "Below we show how this property can be checked in applications. \n", + "\n", + "\n", + "\n", + "\n", + "## A Unique Pairing\n", + "\n", + "Let $Q$ be a conservative intensity matrix on $S$.\n", + "\n", + "Since $Q$ is in $\\linopell$, the operator exponential $e^{tQ}$ is well defined\n", + "as an element of $\\linopell$ for all $t \\geq 0$.\n", + "\n", + "Moreover, by {proof:ref}`ecuc`, the family $(P_t)$ in $\\lL(\\ell_1)$ defined by\n", + "$P_t = e^{tQ}$ defines a UC Markov semigroup on $S$.\n", + "\n", + "(Here, a Markov semigroup $(P_t)$ is both a collection of Markov matrices and a\n", + "collection of operators, as in {eq}`mmismo`.)\n", + "\n", + "The next theorem says that this is the only way UC Markov semigroups can arise.\n", + "\n", + "```{proof:theorem}\n", + ":label: usmg\n", + "\n", + "If $(P_t)$ is a UC Markov semigroup on $\\ell_1$, then there\n", + "exists a conservative intensity matrix $Q$ such that $P_t = e^{tQ}$ for all $t \\geq 0$.\n", + "\n", + "```\n", + "\n", + "```{proof:proof}\n", + "Let $(P_t)$ be a UC Markov semigroup on $\\ell_1$.\n", + "\n", + "Since $(P_t)$ is a UC semigroup on $\\ell_1$, it follows from\n", + "{proof:ref}`ucsgec` that there exists a $Q \\in \\lL(\\ell_1)$ such that \n", + "$P_t = e^{tQ}$ for all $t \\geq 0$.\n", + "\n", + "We need only show that $Q$ is a conservative intensity matrix.\n", + "\n", + "Because $(P_t)$ is a Markov semigroup, $P_t$ is a Markov matrix for all $t$,\n", + "and, since $P_t = e^{tQ}$ for all $t$, it follows that $Q$ is an intensity\n", + "matrix.\n", + "\n", + "We proved this for the case $|S| < \\infty$ in {proof:ref}`intvsmk` and\n", + "one can verify that the same arguments go through when $|S| = \\infty$.\n", + "\n", + "As $Q \\in \\lL(\\ell_1)$, we know that $Q$ is a bounded operator, so $Q$ is a\n", + "conservative intensity matrix. \n", + "```\n", + "\n", + "From {proof:ref}`usmg` we can easily deduce that \n", + "\n", + "* $P_t$ is differentiable at every $t \\geq 0$,\n", + "* $Q$ is the generator of $(P_t)$ and\n", + "* $P_t' = Q P_t = P_t Q$ for all $t \\geq 0$.\n", + "* $P_0' = Q$\n", + "\n", + "In fact these results are just a special case of the claims in {proof:ref}`ucsgec`.\n", + "\n", + "The second last of these results is the Kolmogorov forward and backward equations.\n", + "\n", + "The last of these results shows that we can obtain the intensity matrix $Q$ by differentiating $P_t$ at $t=0$.\n", + "\n", + "```{proof:example}\n", + "Let us consider again the Poisson process $(N_t)$ with rate $\\lambda > 0$ \n", + "in light of the discussion above.\n", + "\n", + "The corresponding semigroup $(P_t)$ is UC and hence there exists a\n", + "conservative intensity matrix $Q$ with $P_t = e^{tQ}$ for all $t \\geq 0$.\n", + "\n", + "This fact can be established by proving UC property and then appealing to\n", + "{proof:ref}`usmg`. \n", + "\n", + "Another alternative, easier in this case, is to supply the intensity matrix\n", + "$Q$ directly and then verify that $P_t = e^{tQ}$ holds.\n", + "\n", + "The semigroup for a Poisson process with rate $\\lambda$ was given in\n", + "{eq}`poissemi` and is repeated here: \n", + "\n", + "$$\n", + " P_t(j, k) \n", + " = \n", + " \\begin{cases}\n", + " e^{-\\lambda t} \\frac{ (\\lambda t)^{k-j} }{(k-j)!} \n", + " & \\text{ if } j \\leq k\n", + " \\\\\n", + " 0 & \\text{ otherwise}\n", + " \\end{cases}\n", + "$$ (poissemi2)\n", + "\n", + "For the intensity matrix we take\n", + "\n", + "$$\n", + " Q := \n", + " \\begin{pmatrix}\n", + " -\\lambda & \\lambda & 0 & 0 & 0 & \\cdots\n", + " \\\\\n", + " 0 & -\\lambda & \\lambda & 0 & 0 & \\cdots\n", + " \\\\\n", + " 0 & 0 & -\\lambda & \\lambda & 0 & \\cdots\n", + " \\\\\n", + " 0 & 0 & 0 & -\\lambda & \\lambda & \\cdots\n", + " \\\\\n", + " \\vdots & \\vdots & \\vdots & \\vdots & \\vdots \n", + " \\end{pmatrix}\n", + "$$ (poissonq)\n", + "\n", + "The form of $Q$ is intuitive: probability flows out of state $i$ and into\n", + "state $i+1$ at the rate $\\lambda$.\n", + "\n", + "It is immediate that $Q$ is an intensity matrix, as claimed.\n", + "\n", + "The exercises ask you to confirm that $Q$ is in $\\lL(\\ell_1)$.\n", + "\n", + "To prove that $P_t = e^{tQ}$ for any $t \\geq 0$, we first decompose $Q$ as $Q\n", + "= \\lambda (K - I)$, where $K$ is defined by \n", + "\n", + "$$\n", + " K(i, j) = \\mathbb 1\\{j = i + 1\\}\n", + "$$\n", + "\n", + "For given $t \\geq 0$, we then have\n", + "\n", + "$$\n", + " e^{tQ}\n", + " = e^{\\lambda t (K-I)}\n", + " = e^{\\lambda t} e^{\\lambda t K}\n", + " = e^{\\lambda t} \n", + " \\sum_{m \\geq 0} \\frac{(\\lambda t K)^m}{m!}\n", + "$$\n", + "\n", + "\n", + "The exercises ask you to verify that, for the powers of $K$, we have $K^m(i,\n", + "j) = \\mathbb 1\\{j = i + m\\}$.\n", + "\n", + "Inserting this expression for $K^m$ leads to \n", + "\n", + "$$\n", + " e^{tQ}(i, j)\n", + " = e^{\\lambda t} \n", + " \\sum_{m \\geq 0} \\frac{(\\lambda t )^m}{m!} \\mathbb 1\\{j = i + m\\}\n", + " = e^{\\lambda t} \n", + " \\sum_{m \\geq 0} \\frac{(\\lambda t )^m}{m!} \\mathbb 1\\{m = j-i\\}\n", + "$$\n", + "\n", + "This is identical to {eq}`poissemi2`.\n", + "\n", + "It now follows that $t \\mapsto P_t \\in \\lL(\\ell_1)$ is differentiable at every\n", + "$t \\geq 0$ and $Q$ is the generator of $(P_t)$, with $P_0' = Q$.\n", + "\n", + "```\n", + "\n", + "\n", + "\n", + "### A Necessary and Sufficient Condition\n", + "\n", + "Our definition of a conservative intensity matrix works for the theory above\n", + "but can be hard to check in appliations and lacks probabilistic intuition. \n", + "\n", + "Fortunately, we have the following simple charcterization.\n", + "\n", + "\n", + "```{proof:lemma} \n", + ":label: scintcon\n", + "\n", + "An intensity matrix $Q$ on $S$ is conservative if and only if $\\sup_x |Q(x,\n", + "x)|$ is finite.\n", + "\n", + "```\n", + "\n", + "The proof is a solved exercise.\n", + "\n", + "\n", + "\n", + "```{proof:example}\n", + ":label: jccs\n", + "\n", + "Recall the jump chain setting where, repeating {eq}`kolbackeq`, we defined $Q$\n", + "via\n", + "\n", + "$$\n", + " Q(x, y) := \\lambda(x) (K(x, y) - I(x, y))\n", + "$$ (kolbackeq_inf)\n", + "\n", + "The function $\\lambda \\colon S \\to \\RR_+$ gives the jump rate at each state,\n", + "while $K$ is the Markov matrix for the embedded discrete time jump chain.\n", + "\n", + "Previously we discussed this in the case where $S$ is finite but there is no\n", + "need to restrict attention to that case.\n", + "\n", + "For general countable $S$, the matrix $Q$ defined in {eq}`kolbackeq_inf` is\n", + "still an intensity matrix.\n", + "\n", + "If we continue to assume that $K(x,x) = 0$ for all $x$, then $Q(x,x) = -\n", + "\\lambda(x)$.\n", + "\n", + "Hence, $Q$ is conservative if and only if $\\sup_x \\lambda(x)$ is finite.\n", + "\n", + "In other words, $Q$ is conservative if the set of jump rates is bounded.\n", + "```\n", + "\n", + "This example shows that requiring $Q$ to be conservative is a relatively mild\n", + "restriction.\n", + "\n", + "\n", + "\n", + "### The Finite State Case\n", + "\n", + "It is immediate from {proof:ref}`scintcon` that every intensity matrix is\n", + "conservative when the state space $S$ is finite.\n", + "\n", + "Hence, in this setting, every intensity matrix $Q$ on $S$ defines a UC Markov\n", + "semigroup $(P_t)$ via $P_t = e^{tQ}$.\n", + "\n", + "Conversely, if $S$ is finite, then any Markov semigroup $(P_t)$ is a UC Markov\n", + "semigroup.\n", + "\n", + "To see this, recall that, as a Markov semigroup, $(P_t)$ satisfies \n", + "$\\lim_{t \\to 0} P_t(x, y) = I(x,y)$ for all $x,y$ in $S$. \n", + "\n", + "In any finite dimensional space, pointwise convergence implies norm\n", + "convergence, so $P_t \\to I$ in operator norm as $t \\to 0$ from above.\n", + "\n", + "As we saw previously, this is enough to ensure that $t \\mapsto P_t$ is norm\n", + "continuous everywhere on $\\RR_+$.\n", + "\n", + "Hence $(P_t)$ is a UC Markov semigroup, as claimed.\n", + "\n", + "Combining these results with {proof:ref}`usmg`, we conclude that, when $S$ is\n", + "finite, there is a one-to-one correspondence between Markov semigroups and\n", + "intensity matrices.\n", + "\n", + "\n", + "\n", + "## Beyond Bounded Intensity Matrices\n", + "\n", + "If we do run into an application where an intensity matrix $Q$ is not\n", + "conservative, what might we expect?\n", + "\n", + "In this scenario, we can at least hope that $Q$ is the generator of a $C_0$\n", + "semigroup.\n", + "\n", + "Since $Q$ is an intensity matrix, we can be sure that this semigroup will be a\n", + "Markov semigroup.\n", + "\n", + "To know when $Q$ will be the generator of a $C_0$\n", + "semigroup, we need to look to the [Hille-Yoshida\n", + "Theorem](https://en.wikipedia.org/wiki/Hille%E2%80%93Yosida_theorem) and\n", + "sufficient conditions derived from it.\n", + "\n", + "\n", + "While we omit a detailed treatment, it is worth noting that the issue is\n", + "linked to explosions.\n", + "\n", + "To see the connection, recall that some initial value problems do not lead to a\n", + "valid solution defined for all $t \\in \\RR_+$.\n", + "\n", + "An example is the scalar problem $x'_t = 1 + x_t^2$, which has solution $x_t =\n", + "\\tan (t - c)$ for some constant $c$.\n", + "\n", + "But this solution equals $+\\infty$ for $t \\geq c + \\pi/2$.\n", + "\n", + "The problem is that the time path explodes to infinity in finite time.\n", + "\n", + "The same issue can occur for Markov processes, if jump rates grow sufficiently\n", + "quickly.\n", + "\n", + "For more discussion, see, for example, Section 2.7 of {cite}`norris1998markov`.\n", + "\n", + "\n", + "## Exercises\n", + "\n", + "### Exercise 1\n", + "\n", + "Let $P$ be a Markov matrix on $S$ and identify it with the linear operator in\n", + "{eq}`mmismo`. Verify the claims in {eq}`propp`.\n", + "\n", + "### Exercise 2\n", + "\n", + "Prove the claim in {proof:ref}`scintcon`.\n", + "\n", + "### Exercise 3\n", + "\n", + "Confirm that $Q$ defined in {eq}`poissonq` induces a bounded linear operator on\n", + "$\\ell_1$ via {eq}`imislo`.\n", + "\n", + "\n", + "### Exercise 4\n", + "\n", + "Let $K$ be defined on $\\ZZ_+ \\times \\ZZ_+$ by $K(i, j) = \\mathbb 1\\{j = i + 1\\}$. \n", + "\n", + "Show that, with $K^m$ representing the $m$-th matrix product of $K$ with itself, \n", + "we have $K^m(i, j) = \\mathbb 1\\{j = i + m\\}$ for any $i, j \\in \\ZZ_+$.\n", + " \n", + "\n", + "## Solutions\n", + "\n", + "### Solution to Exercise 1\n", + "\n", + "To determine the norm of $P$, we use the definition in {eq}`norml`.\n", + "\n", + "If $f \\in \\ell_1$ and $\\| f \\| \\leq 1$, then \n", + "\n", + "$$\n", + " \\| f P \\| \n", + " \\leq \\sum_y \\sum_x |f(x)| P(x, y)\n", + " = \\sum_x |f(x)| \\sum_y P(x, y)\n", + " = \\sum_x |f(x)| \n", + " = \\| f \\|\n", + "$$\n", + "\n", + "Hence $\\| P \\| \\leq 1$. \n", + "\n", + "To see that equality holds we can repeat this argument with $f \\geq 0$,\n", + "obtaining $\\| fP \\| = \\|f\\|$.\n", + "\n", + "Now pick any $\\phi \\in \\dD$. \n", + "\n", + "Clearly $\\phi P \\geq 0$, and \n", + "\n", + "$$\n", + " \\sum_y (\\phi P)(y)\n", + " \\sum_y \\sum_x \\phi (x) P(x, y)\n", + " \\sum_x \\phi (x) \\sum_y P(x, y)\n", + " = 1\n", + "$$\n", + "\n", + "Hence $\\phi P \\in \\dD$ as claimed.\n", + "\n", + "### Solution to Exercise 2\n", + "\n", + "\n", + "Here is one solution.\n", + "\n", + "Let $Q$ be an intensity matrix on $S$.\n", + "\n", + "Suppose first that $m := \\sup_x |Q(x,x)|$ is finite.\n", + "\n", + "Set $\\hat P := I + Q / m$.\n", + "\n", + "It is not hard to check that $\\hat P$ is a Markov matrix and that $Q = m( \\hat\n", + "P - I)$.\n", + "\n", + "Since $\\hat P$ is a Markov matrix, it induces a bounded linear operator on\n", + "$\\ell_1$ via {eq}`mmismo`. \n", + "\n", + "As $\\lL(\\ell_1)$ is a linear space, we see that $Q$ is likewise in\n", + "$\\lL(\\ell_1)$. \n", + "\n", + "In particular, $Q$ is a bounded operator, and hence conservative.\n", + "\n", + "Next, suppose that $Q$ is conservative and yet $\\sup_x |Q(x,x)|$ is infinite.\n", + "\n", + "Choose $x \\in S$ such that $|Q(x, x)| > \\| Q\\|$\n", + "\n", + "Let $f \\in \\ell_1$ be defined by $f(z) = \\mathbb 1\\{z = x\\}$.\n", + "\n", + "Since $\\|f\\| = 1$, we have\n", + "\n", + "$$\n", + " \\| Q \\| \n", + " \\geq \\| f Q \\|\n", + " = \\sum_y \\left| \\sum_z f(z) Q(z, y) \\right|\n", + " = \\sum_y | Q(x, y) |\n", + " \\geq | Q(x, x) |\n", + "$$\n", + "\n", + "Contradiction.\n", + "\n", + "### Solution to Exercise 3\n", + "\n", + "Linearity is obvious so we focus on boundedness.\n", + "\n", + "For any $f \\in \\ell_1$ and this choice of $Q$, we have\n", + "\n", + "$$\n", + " \\sum_y |(fQ)(y)|\n", + " \\leq \\sum_y \\sum_x |f(x) Q(x, y)|\n", + " \\leq \\lambda \\sum_y \\sum_x |f(y) - f(y+1)|\n", + "$$\n", + "\n", + "Applying the triangle inequality, we see that the right hand side is dominated\n", + "by $2 \\lambda \\| f\\|$.\n", + "\n", + "Hence $\\| fQ \\| \\leq 2 \\lambda \\|f\\|$, which implies that $Q \\in \\lL(\\ell_1)$\n", + "as required.\n", + "\n", + "\n", + "\n", + "### Solution to Exercise 4\n", + "\n", + "The statement $K^m(i, j) = \\mathbb 1\\{j = i + m\\}$ holds by definition when\n", + "$m=1$.\n", + "\n", + "Now suppose it holds at arbitrary $m$.\n", + "\n", + "We then have, by definition of composition (matrix multiplication),\n", + "\n", + "$$\n", + " K^{m+1}(i, j)\n", + " = \\sum_n K(i, n) K^m (n, j)\n", + " = \\sum_n K(i, n) \\mathbb 1\\{j = i + m\\}\n", + " = K(i, m-j)\n", + "$$\n", + "\n", + "Applying the definition $K(i, j) = \\mathbb 1\\{j = i + 1\\}$ completes verification of the claim." + ] + } + ], + "metadata": { + "jupytext": { + "formats": "ipynb,md:myst", + "text_representation": { + "extension": ".md", + "format_name": "myst", + "format_version": "0.9", + "jupytext_version": "1.5.0" + } + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.7" + }, + "source_map": [ + 13 + ] + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file diff --git a/_sources/uc_mc_semigroups.md b/_sources/uc_mc_semigroups.md new file mode 100644 index 0000000..d60aed0 --- /dev/null +++ b/_sources/uc_mc_semigroups.md @@ -0,0 +1,493 @@ +--- +jupytext: + formats: ipynb,md:myst + text_representation: + extension: .md + format_name: myst + format_version: '0.9' + jupytext_version: 1.5.0 +kernelspec: + display_name: Python 3 + language: python + name: python3 +--- + + +# UC Markov Semigroups + +## Overview + +In our previous lecture we covered some of the general theory of continuous +operator semigroups. + +Next we translate these results into the setting of Markov semigroups. + +The main aim is to give an exact one-to-one correspondence between UC +Markov semigroups and "conservative" intensity matrices on a countable set $S$. + +Conservativeness is defined below and relates to "nonexplosiveness" of the +associated Markov matrix. + +We will also give a brief discussion of intensity matricies that fall +outside this class, along with the processes they generate. + + +## Notation and Terminology + +Let $S$ be an arbitrary countable set. + +Let $\ell_1$ be the Banach space of **summable functions** on $S$; that is, all $g \, \colon S \to \RR$ with + +$$ + \| g \| := \sum_x |g(x)| < \infty +$$ + +Note that $\dD$, the set of distributions on $S$, is contained in $\ell_1$. + +Each Markov matrix $P$ on $S$ can and will be identifed with a linear operator +$f \mapsto fP$ on $\ell_1$ via + +$$ + (fP)(y) = \sum_x f(x) P(x, y) + \qquad (f \in \ell_1, \; y \in S) +$$ (mmismo) + +To be consistent with earlier notation, we are writing the argument of $P$ to +the left and applying $P$ to it as if premultiplying $P$ by a row vector. + +In the exercises you are asked to verify that {eq}`mmismo` defines +a bounded linear operator on $\ell_1$ such that + +$$ + \|P\| = 1 \text{ and } \phi P \in \dD \text{ whenever } \phi \in \dD +$$ (propp) + +Note that composition of $P$ with itself is equivalent to powers of the matrix +under matrix multiplication. + +For an intensity matrix $Q$ on $S$ we can try to introduce the associated +operator analogously, via + +$$ + (fQ)(y) = \sum_x f(x) Q(x, y) + \qquad (f \in \ell_1, \; y \in S) +$$ (imislo) + +However, the sum in {eq}`imislo` is not always well defined.[^fnim] + +[^fnim]: Previously, we introduced the notion of an intensity matrix when $S$ + is finite and the definition is essentially unchanged in the current + setting. In particular, $Q \colon S \times S \to \RR$ is called an + **intensity matrix** if $Q$ has zero row sums and $Q(x, y) \geq 0$ whenever + $x \not= y$. + +We say that an intensity matrix $Q$ is **conservative** if the sum in +{eq}`imislo` is well defined at all $y$ and, in addition, the mapping $f +\mapsto fQ$ in {eq}`imislo` is a bounded linear operator on $\ell_1$. + +Below we show how this property can be checked in applications. + + + + +## A Unique Pairing + +Let $Q$ be a conservative intensity matrix on $S$. + +Since $Q$ is in $\linopell$, the operator exponential $e^{tQ}$ is well defined +as an element of $\linopell$ for all $t \geq 0$. + +Moreover, by {proof:ref}`ecuc`, the family $(P_t)$ in $\lL(\ell_1)$ defined by +$P_t = e^{tQ}$ defines a UC Markov semigroup on $S$. + +(Here, a Markov semigroup $(P_t)$ is both a collection of Markov matrices and a +collection of operators, as in {eq}`mmismo`.) + +The next theorem says that this is the only way UC Markov semigroups can arise. + +```{proof:theorem} +:label: usmg + +If $(P_t)$ is a UC Markov semigroup on $\ell_1$, then there +exists a conservative intensity matrix $Q$ such that $P_t = e^{tQ}$ for all $t \geq 0$. + +``` + +```{proof:proof} +Let $(P_t)$ be a UC Markov semigroup on $\ell_1$. + +Since $(P_t)$ is a UC semigroup on $\ell_1$, it follows from +{proof:ref}`ucsgec` that there exists a $Q \in \lL(\ell_1)$ such that +$P_t = e^{tQ}$ for all $t \geq 0$. + +We need only show that $Q$ is a conservative intensity matrix. + +Because $(P_t)$ is a Markov semigroup, $P_t$ is a Markov matrix for all $t$, +and, since $P_t = e^{tQ}$ for all $t$, it follows that $Q$ is an intensity +matrix. + +We proved this for the case $|S| < \infty$ in {proof:ref}`intvsmk` and +one can verify that the same arguments go through when $|S| = \infty$. + +As $Q \in \lL(\ell_1)$, we know that $Q$ is a bounded operator, so $Q$ is a +conservative intensity matrix. +``` + +From {proof:ref}`usmg` we can easily deduce that + +* $P_t$ is differentiable at every $t \geq 0$, +* $Q$ is the generator of $(P_t)$ and +* $P_t' = Q P_t = P_t Q$ for all $t \geq 0$. +* $P_0' = Q$ + +In fact these results are just a special case of the claims in {proof:ref}`ucsgec`. + +The second last of these results is the Kolmogorov forward and backward equations. + +The last of these results shows that we can obtain the intensity matrix $Q$ by differentiating $P_t$ at $t=0$. + +```{proof:example} +Let us consider again the Poisson process $(N_t)$ with rate $\lambda > 0$ +in light of the discussion above. + +The corresponding semigroup $(P_t)$ is UC and hence there exists a +conservative intensity matrix $Q$ with $P_t = e^{tQ}$ for all $t \geq 0$. + +This fact can be established by proving UC property and then appealing to +{proof:ref}`usmg`. + +Another alternative, easier in this case, is to supply the intensity matrix +$Q$ directly and then verify that $P_t = e^{tQ}$ holds. + +The semigroup for a Poisson process with rate $\lambda$ was given in +{eq}`poissemi` and is repeated here: + +$$ + P_t(j, k) + = + \begin{cases} + e^{-\lambda t} \frac{ (\lambda t)^{k-j} }{(k-j)!} + & \text{ if } j \leq k + \\ + 0 & \text{ otherwise} + \end{cases} +$$ (poissemi2) + +For the intensity matrix we take + +$$ + Q := + \begin{pmatrix} + -\lambda & \lambda & 0 & 0 & 0 & \cdots + \\ + 0 & -\lambda & \lambda & 0 & 0 & \cdots + \\ + 0 & 0 & -\lambda & \lambda & 0 & \cdots + \\ + 0 & 0 & 0 & -\lambda & \lambda & \cdots + \\ + \vdots & \vdots & \vdots & \vdots & \vdots + \end{pmatrix} +$$ (poissonq) + +The form of $Q$ is intuitive: probability flows out of state $i$ and into +state $i+1$ at the rate $\lambda$. + +It is immediate that $Q$ is an intensity matrix, as claimed. + +The exercises ask you to confirm that $Q$ is in $\lL(\ell_1)$. + +To prove that $P_t = e^{tQ}$ for any $t \geq 0$, we first decompose $Q$ as $Q += \lambda (K - I)$, where $K$ is defined by + +$$ + K(i, j) = \mathbb 1\{j = i + 1\} +$$ + +For given $t \geq 0$, we then have + +$$ + e^{tQ} + = e^{\lambda t (K-I)} + = e^{\lambda t} e^{\lambda t K} + = e^{\lambda t} + \sum_{m \geq 0} \frac{(\lambda t K)^m}{m!} +$$ + + +The exercises ask you to verify that, for the powers of $K$, we have $K^m(i, +j) = \mathbb 1\{j = i + m\}$. + +Inserting this expression for $K^m$ leads to + +$$ + e^{tQ}(i, j) + = e^{\lambda t} + \sum_{m \geq 0} \frac{(\lambda t )^m}{m!} \mathbb 1\{j = i + m\} + = e^{\lambda t} + \sum_{m \geq 0} \frac{(\lambda t )^m}{m!} \mathbb 1\{m = j-i\} +$$ + +This is identical to {eq}`poissemi2`. + +It now follows that $t \mapsto P_t \in \lL(\ell_1)$ is differentiable at every +$t \geq 0$ and $Q$ is the generator of $(P_t)$, with $P_0' = Q$. + +``` + + + +### A Necessary and Sufficient Condition + +Our definition of a conservative intensity matrix works for the theory above +but can be hard to check in appliations and lacks probabilistic intuition. + +Fortunately, we have the following simple charcterization. + + +```{proof:lemma} +:label: scintcon + +An intensity matrix $Q$ on $S$ is conservative if and only if $\sup_x |Q(x, +x)|$ is finite. + +``` + +The proof is a solved exercise. + + + +```{proof:example} +:label: jccs + +Recall the jump chain setting where, repeating {eq}`kolbackeq`, we defined $Q$ +via + +$$ + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +$$ (kolbackeq_inf) + +The function $\lambda \colon S \to \RR_+$ gives the jump rate at each state, +while $K$ is the Markov matrix for the embedded discrete time jump chain. + +Previously we discussed this in the case where $S$ is finite but there is no +need to restrict attention to that case. + +For general countable $S$, the matrix $Q$ defined in {eq}`kolbackeq_inf` is +still an intensity matrix. + +If we continue to assume that $K(x,x) = 0$ for all $x$, then $Q(x,x) = - +\lambda(x)$. + +Hence, $Q$ is conservative if and only if $\sup_x \lambda(x)$ is finite. + +In other words, $Q$ is conservative if the set of jump rates is bounded. +``` + +This example shows that requiring $Q$ to be conservative is a relatively mild +restriction. + + + +### The Finite State Case + +It is immediate from {proof:ref}`scintcon` that every intensity matrix is +conservative when the state space $S$ is finite. + +Hence, in this setting, every intensity matrix $Q$ on $S$ defines a UC Markov +semigroup $(P_t)$ via $P_t = e^{tQ}$. + +Conversely, if $S$ is finite, then any Markov semigroup $(P_t)$ is a UC Markov +semigroup. + +To see this, recall that, as a Markov semigroup, $(P_t)$ satisfies +$\lim_{t \to 0} P_t(x, y) = I(x,y)$ for all $x,y$ in $S$. + +In any finite dimensional space, pointwise convergence implies norm +convergence, so $P_t \to I$ in operator norm as $t \to 0$ from above. + +As we saw previously, this is enough to ensure that $t \mapsto P_t$ is norm +continuous everywhere on $\RR_+$. + +Hence $(P_t)$ is a UC Markov semigroup, as claimed. + +Combining these results with {proof:ref}`usmg`, we conclude that, when $S$ is +finite, there is a one-to-one correspondence between Markov semigroups and +intensity matrices. + + + +## Beyond Bounded Intensity Matrices + +If we do run into an application where an intensity matrix $Q$ is not +conservative, what might we expect? + +In this scenario, we can at least hope that $Q$ is the generator of a $C_0$ +semigroup. + +Since $Q$ is an intensity matrix, we can be sure that this semigroup will be a +Markov semigroup. + +To know when $Q$ will be the generator of a $C_0$ +semigroup, we need to look to the [Hille-Yoshida +Theorem](https://en.wikipedia.org/wiki/Hille%E2%80%93Yosida_theorem) and +sufficient conditions derived from it. + + +While we omit a detailed treatment, it is worth noting that the issue is +linked to explosions. + +To see the connection, recall that some initial value problems do not lead to a +valid solution defined for all $t \in \RR_+$. + +An example is the scalar problem $x'_t = 1 + x_t^2$, which has solution $x_t = +\tan (t - c)$ for some constant $c$. + +But this solution equals $+\infty$ for $t \geq c + \pi/2$. + +The problem is that the time path explodes to infinity in finite time. + +The same issue can occur for Markov processes, if jump rates grow sufficiently +quickly. + +For more discussion, see, for example, Section 2.7 of {cite}`norris1998markov`. + + +## Exercises + +### Exercise 1 + +Let $P$ be a Markov matrix on $S$ and identify it with the linear operator in +{eq}`mmismo`. Verify the claims in {eq}`propp`. + +### Exercise 2 + +Prove the claim in {proof:ref}`scintcon`. + +### Exercise 3 + +Confirm that $Q$ defined in {eq}`poissonq` induces a bounded linear operator on +$\ell_1$ via {eq}`imislo`. + + +### Exercise 4 + +Let $K$ be defined on $\ZZ_+ \times \ZZ_+$ by $K(i, j) = \mathbb 1\{j = i + 1\}$. + +Show that, with $K^m$ representing the $m$-th matrix product of $K$ with itself, +we have $K^m(i, j) = \mathbb 1\{j = i + m\}$ for any $i, j \in \ZZ_+$. + + +## Solutions + +### Solution to Exercise 1 + +To determine the norm of $P$, we use the definition in {eq}`norml`. + +If $f \in \ell_1$ and $\| f \| \leq 1$, then + +$$ + \| f P \| + \leq \sum_y \sum_x |f(x)| P(x, y) + = \sum_x |f(x)| \sum_y P(x, y) + = \sum_x |f(x)| + = \| f \| +$$ + +Hence $\| P \| \leq 1$. + +To see that equality holds we can repeat this argument with $f \geq 0$, +obtaining $\| fP \| = \|f\|$. + +Now pick any $\phi \in \dD$. + +Clearly $\phi P \geq 0$, and + +$$ + \sum_y (\phi P)(y) + \sum_y \sum_x \phi (x) P(x, y) + \sum_x \phi (x) \sum_y P(x, y) + = 1 +$$ + +Hence $\phi P \in \dD$ as claimed. + +### Solution to Exercise 2 + + +Here is one solution. + +Let $Q$ be an intensity matrix on $S$. + +Suppose first that $m := \sup_x |Q(x,x)|$ is finite. + +Set $\hat P := I + Q / m$. + +It is not hard to check that $\hat P$ is a Markov matrix and that $Q = m( \hat +P - I)$. + +Since $\hat P$ is a Markov matrix, it induces a bounded linear operator on +$\ell_1$ via {eq}`mmismo`. + +As $\lL(\ell_1)$ is a linear space, we see that $Q$ is likewise in +$\lL(\ell_1)$. + +In particular, $Q$ is a bounded operator, and hence conservative. + +Next, suppose that $Q$ is conservative and yet $\sup_x |Q(x,x)|$ is infinite. + +Choose $x \in S$ such that $|Q(x, x)| > \| Q\|$ + +Let $f \in \ell_1$ be defined by $f(z) = \mathbb 1\{z = x\}$. + +Since $\|f\| = 1$, we have + +$$ + \| Q \| + \geq \| f Q \| + = \sum_y \left| \sum_z f(z) Q(z, y) \right| + = \sum_y | Q(x, y) | + \geq | Q(x, x) | +$$ + +Contradiction. + +### Solution to Exercise 3 + +Linearity is obvious so we focus on boundedness. + +For any $f \in \ell_1$ and this choice of $Q$, we have + +$$ + \sum_y |(fQ)(y)| + \leq \sum_y \sum_x |f(x) Q(x, y)| + \leq \lambda \sum_y \sum_x |f(y) - f(y+1)| +$$ + +Applying the triangle inequality, we see that the right hand side is dominated +by $2 \lambda \| f\|$. + +Hence $\| fQ \| \leq 2 \lambda \|f\|$, which implies that $Q \in \lL(\ell_1)$ +as required. + + + +### Solution to Exercise 4 + +The statement $K^m(i, j) = \mathbb 1\{j = i + m\}$ holds by definition when +$m=1$. + +Now suppose it holds at arbitrary $m$. + +We then have, by definition of composition (matrix multiplication), + +$$ + K^{m+1}(i, j) + = \sum_n K(i, n) K^m (n, j) + = \sum_n K(i, n) \mathbb 1\{j = i + m\} + = K(i, m-j) +$$ + +Applying the definition $K(i, j) = \mathbb 1\{j = i + 1\}$ completes verification of the claim. + + diff --git a/_sources/zreferences.md b/_sources/zreferences.md new file mode 100644 index 0000000..7c6cace --- /dev/null +++ b/_sources/zreferences.md @@ -0,0 +1,7 @@ +# Bibliography + + +## References + +```{bibliography} references.bib +``` diff --git a/_static/basic.css b/_static/basic.css new file mode 100644 index 0000000..9f93524 --- /dev/null +++ b/_static/basic.css @@ -0,0 +1,768 @@ +/* + * basic.css + * ~~~~~~~~~ + * + * Sphinx stylesheet -- basic theme. + * + * :copyright: Copyright 2007-2020 by the Sphinx team, see AUTHORS. + * :license: BSD, see LICENSE for details. + * + */ + +/* -- main layout ----------------------------------------------------------- */ + +div.clearer { + clear: both; +} + +/* -- relbar ---------------------------------------------------------------- */ + +div.related { + width: 100%; + font-size: 90%; +} + +div.related h3 { + display: none; +} + +div.related ul { + margin: 0; + padding: 0 0 0 10px; + list-style: none; +} + +div.related li { + display: inline; +} + +div.related li.right { + float: right; + margin-right: 5px; +} + +/* -- sidebar --------------------------------------------------------------- */ + +div.sphinxsidebarwrapper { + padding: 10px 5px 0 10px; +} + +div.sphinxsidebar { + float: left; + width: 270px; + margin-left: -100%; + font-size: 90%; + word-wrap: break-word; + overflow-wrap : break-word; +} + +div.sphinxsidebar ul { + list-style: none; +} + +div.sphinxsidebar ul ul, +div.sphinxsidebar ul.want-points { + margin-left: 20px; + list-style: square; +} + +div.sphinxsidebar ul ul { + margin-top: 0; + margin-bottom: 0; +} + +div.sphinxsidebar form { + margin-top: 10px; +} + +div.sphinxsidebar input { + border: 1px solid #98dbcc; + font-family: sans-serif; + font-size: 1em; +} + +div.sphinxsidebar #searchbox form.search { + overflow: hidden; +} + +div.sphinxsidebar #searchbox input[type="text"] { + float: left; + width: 80%; + padding: 0.25em; + box-sizing: border-box; +} + +div.sphinxsidebar #searchbox input[type="submit"] { + float: left; + width: 20%; + border-left: none; + padding: 0.25em; + box-sizing: border-box; +} + + +img { + border: 0; + max-width: 100%; +} + +/* -- search page ----------------------------------------------------------- */ + +ul.search { + margin: 10px 0 0 20px; + padding: 0; +} + +ul.search li { + padding: 5px 0 5px 20px; + background-image: url(file.png); + background-repeat: no-repeat; + background-position: 0 7px; +} + +ul.search li a { + font-weight: bold; +} + +ul.search li div.context { + color: #888; + margin: 2px 0 0 30px; + text-align: left; +} + +ul.keywordmatches li.goodmatch a { + font-weight: bold; +} + +/* -- index page ------------------------------------------------------------ */ + +table.contentstable { + width: 90%; + margin-left: auto; + margin-right: auto; +} + +table.contentstable p.biglink { + line-height: 150%; +} + +a.biglink { + font-size: 1.3em; +} + +span.linkdescr { + font-style: italic; + padding-top: 5px; + font-size: 90%; +} + +/* -- general index --------------------------------------------------------- */ + +table.indextable { + width: 100%; +} + +table.indextable td { + text-align: left; + vertical-align: top; +} + +table.indextable ul { + margin-top: 0; + margin-bottom: 0; + list-style-type: none; +} + +table.indextable > tbody > tr > td > ul { + padding-left: 0em; +} + +table.indextable tr.pcap { + height: 10px; +} + +table.indextable tr.cap { + margin-top: 10px; + background-color: #f2f2f2; +} + +img.toggler { + margin-right: 3px; + margin-top: 3px; + cursor: pointer; +} + +div.modindex-jumpbox { + border-top: 1px solid #ddd; + border-bottom: 1px solid #ddd; + margin: 1em 0 1em 0; + padding: 0.4em; +} + +div.genindex-jumpbox { + border-top: 1px solid #ddd; + border-bottom: 1px solid #ddd; + margin: 1em 0 1em 0; + padding: 0.4em; +} + +/* -- domain module index --------------------------------------------------- */ + +table.modindextable td { + padding: 2px; + border-collapse: collapse; +} + +/* -- general body styles --------------------------------------------------- */ + +div.body { + min-width: 450px; + max-width: 800px; +} + +div.body p, div.body dd, div.body li, div.body blockquote { + -moz-hyphens: auto; + -ms-hyphens: auto; + -webkit-hyphens: auto; + hyphens: auto; +} + +a.headerlink { + visibility: hidden; +} + +a.brackets:before, +span.brackets > a:before{ + content: "["; +} + +a.brackets:after, +span.brackets > a:after { + content: "]"; +} + +h1:hover > a.headerlink, +h2:hover > a.headerlink, +h3:hover > a.headerlink, +h4:hover > a.headerlink, +h5:hover > a.headerlink, +h6:hover > a.headerlink, +dt:hover > a.headerlink, +caption:hover > a.headerlink, +p.caption:hover > a.headerlink, +div.code-block-caption:hover > a.headerlink { + visibility: visible; +} + +div.body p.caption { + text-align: inherit; +} + +div.body td { + text-align: left; +} + +.first { + margin-top: 0 !important; +} + +p.rubric { + margin-top: 30px; + font-weight: bold; +} + +img.align-left, .figure.align-left, object.align-left { + clear: left; + float: left; + margin-right: 1em; +} + +img.align-right, .figure.align-right, object.align-right { + clear: right; + float: right; + margin-left: 1em; +} + +img.align-center, .figure.align-center, object.align-center { + display: block; + margin-left: auto; + margin-right: auto; +} + +img.align-default, .figure.align-default { + display: block; + margin-left: auto; + margin-right: auto; +} + +.align-left { + text-align: left; +} + +.align-center { + text-align: center; +} + +.align-default { + text-align: center; +} + +.align-right { + text-align: right; +} + +/* -- sidebars -------------------------------------------------------------- */ + +div.sidebar { + margin: 0 0 0.5em 1em; + border: 1px solid #ddb; + padding: 7px 7px 0 7px; + background-color: #ffe; + width: 40%; + float: right; +} + +p.sidebar-title { + font-weight: bold; +} + +/* -- topics ---------------------------------------------------------------- */ + +div.topic { + border: 1px solid #ccc; + padding: 7px 7px 0 7px; + margin: 10px 0 10px 0; +} + +p.topic-title { + font-size: 1.1em; + font-weight: bold; + margin-top: 10px; +} + +/* -- admonitions ----------------------------------------------------------- */ + +div.admonition { + margin-top: 10px; + margin-bottom: 10px; + padding: 7px; +} + +div.admonition dt { + font-weight: bold; +} + +div.admonition dl { + margin-bottom: 0; +} + +p.admonition-title { + margin: 0px 10px 5px 0px; + font-weight: bold; +} + +div.body p.centered { + text-align: center; + margin-top: 25px; +} + +/* -- tables ---------------------------------------------------------------- */ + +table.docutils { + border: 0; + border-collapse: collapse; +} + +table.align-center { + margin-left: auto; + margin-right: auto; +} + +table.align-default { + margin-left: auto; + margin-right: auto; +} + +table caption span.caption-number { + font-style: italic; +} + +table caption span.caption-text { +} + +table.docutils td, table.docutils th { + padding: 1px 8px 1px 5px; + border-top: 0; + border-left: 0; + border-right: 0; + border-bottom: 1px solid #aaa; +} + +table.footnote td, table.footnote th { + border: 0 !important; +} + +th { + text-align: left; + padding-right: 5px; +} + +table.citation { + border-left: solid 1px gray; + margin-left: 1px; +} + +table.citation td { + border-bottom: none; +} + +th > p:first-child, +td > p:first-child { + margin-top: 0px; +} + +th > p:last-child, +td > p:last-child { + margin-bottom: 0px; +} + +/* -- figures --------------------------------------------------------------- */ + +div.figure { + margin: 0.5em; + padding: 0.5em; +} + +div.figure p.caption { + padding: 0.3em; +} + +div.figure p.caption span.caption-number { + font-style: italic; +} + +div.figure p.caption span.caption-text { +} + +/* -- field list styles ----------------------------------------------------- */ + +table.field-list td, table.field-list th { + border: 0 !important; +} + +.field-list ul { + margin: 0; + padding-left: 1em; +} + +.field-list p { + margin: 0; +} + +.field-name { + -moz-hyphens: manual; + -ms-hyphens: manual; + -webkit-hyphens: manual; + hyphens: manual; +} + +/* -- hlist styles ---------------------------------------------------------- */ + +table.hlist td { + vertical-align: top; +} + + +/* -- other body styles ----------------------------------------------------- */ + +ol.arabic { + list-style: decimal; +} + +ol.loweralpha { + list-style: lower-alpha; +} + +ol.upperalpha { + list-style: upper-alpha; +} + +ol.lowerroman { + list-style: lower-roman; +} + +ol.upperroman { + list-style: upper-roman; +} + +li > p:first-child { + margin-top: 0px; +} + +li > p:last-child { + margin-bottom: 0px; +} + +dl.footnote > dt, +dl.citation > dt { + float: left; +} + +dl.footnote > dd, +dl.citation > dd { + margin-bottom: 0em; +} + +dl.footnote > dd:after, +dl.citation > dd:after { + content: ""; + clear: both; +} + +dl.field-list { + display: grid; + grid-template-columns: fit-content(30%) auto; +} + +dl.field-list > dt { + font-weight: bold; + word-break: break-word; + padding-left: 0.5em; + padding-right: 5px; +} + +dl.field-list > dt:after { + content: ":"; +} + +dl.field-list > dd { + padding-left: 0.5em; + margin-top: 0em; + margin-left: 0em; + margin-bottom: 0em; +} + +dl { + margin-bottom: 15px; +} + +dd > p:first-child { + margin-top: 0px; +} + +dd ul, dd table { + margin-bottom: 10px; +} + +dd { + margin-top: 3px; + margin-bottom: 10px; + margin-left: 30px; +} + +dt:target, span.highlighted { + background-color: #fbe54e; +} + +rect.highlighted { + fill: #fbe54e; +} + +dl.glossary dt { + font-weight: bold; + font-size: 1.1em; +} + +.optional { + font-size: 1.3em; +} + +.sig-paren { + font-size: larger; +} + +.versionmodified { + font-style: italic; +} + +.system-message { + background-color: #fda; + padding: 5px; + border: 3px solid red; +} + +.footnote:target { + background-color: #ffa; +} + +.line-block { + display: block; + margin-top: 1em; + margin-bottom: 1em; +} + +.line-block .line-block { + margin-top: 0; + margin-bottom: 0; + margin-left: 1.5em; +} + +.guilabel, .menuselection { + font-family: sans-serif; +} + +.accelerator { + text-decoration: underline; +} + +.classifier { + font-style: oblique; +} + +.classifier:before { + font-style: normal; + margin: 0.5em; + content: ":"; +} + +abbr, acronym { + border-bottom: dotted 1px; + cursor: help; +} + +/* -- code displays --------------------------------------------------------- */ + +pre { + overflow: auto; + overflow-y: hidden; /* fixes display issues on Chrome browsers */ +} + +span.pre { + -moz-hyphens: none; + -ms-hyphens: none; + -webkit-hyphens: none; + hyphens: none; +} + +td.linenos pre { + padding: 5px 0px; + border: 0; + background-color: transparent; + color: #aaa; +} + +table.highlighttable { + margin-left: 0.5em; +} + +table.highlighttable td { + padding: 0 0.5em 0 0.5em; +} + +div.code-block-caption { + padding: 2px 5px; + font-size: small; +} + +div.code-block-caption code { + background-color: transparent; +} + +div.code-block-caption + div > div.highlight > pre { + margin-top: 0; +} + +div.doctest > div.highlight span.gp { /* gp: Generic.Prompt */ + user-select: none; +} + +div.code-block-caption span.caption-number { + padding: 0.1em 0.3em; + font-style: italic; +} + +div.code-block-caption span.caption-text { +} + +div.literal-block-wrapper { + padding: 1em 1em 0; +} + +div.literal-block-wrapper div.highlight { + margin: 0; +} + +code.descname { + background-color: transparent; + font-weight: bold; + font-size: 1.2em; +} + +code.descclassname { + background-color: transparent; +} + +code.xref, a code { + background-color: transparent; + font-weight: bold; +} + +h1 code, h2 code, h3 code, h4 code, h5 code, h6 code { + background-color: transparent; +} + +.viewcode-link { + float: right; +} + +.viewcode-back { + float: right; + font-family: sans-serif; +} + +div.viewcode-block:target { + margin: -1px -10px; + padding: 0 10px; +} + +/* -- math display ---------------------------------------------------------- */ + +img.math { + vertical-align: middle; +} + +div.body div.math p { + text-align: center; +} + +span.eqno { + float: right; +} + +span.eqno a.headerlink { + position: relative; + left: 0px; + z-index: 1; +} + +div.math:hover a.headerlink { + visibility: visible; +} + +/* -- printout stylesheet --------------------------------------------------- */ + +@media print { + div.document, + div.documentwrapper, + div.bodywrapper { + margin: 0 !important; + width: 100%; + } + + div.sphinxsidebar, + div.related, + div.footer, + #top-link { + display: none; + } +} \ No newline at end of file diff --git a/_static/clipboard.min.js b/_static/clipboard.min.js new file mode 100644 index 0000000..02c549e --- /dev/null +++ b/_static/clipboard.min.js @@ -0,0 +1,7 @@ +/*! + * clipboard.js v2.0.4 + * https://zenorocha.github.io/clipboard.js + * + * Licensed MIT © Zeno Rocha + */ +!function(t,e){"object"==typeof exports&&"object"==typeof module?module.exports=e():"function"==typeof define&&define.amd?define([],e):"object"==typeof exports?exports.ClipboardJS=e():t.ClipboardJS=e()}(this,function(){return function(n){var o={};function r(t){if(o[t])return o[t].exports;var e=o[t]={i:t,l:!1,exports:{}};return n[t].call(e.exports,e,e.exports,r),e.l=!0,e.exports}return r.m=n,r.c=o,r.d=function(t,e,n){r.o(t,e)||Object.defineProperty(t,e,{enumerable:!0,get:n})},r.r=function(t){"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(t,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(t,"__esModule",{value:!0})},r.t=function(e,t){if(1&t&&(e=r(e)),8&t)return e;if(4&t&&"object"==typeof e&&e&&e.__esModule)return e;var n=Object.create(null);if(r.r(n),Object.defineProperty(n,"default",{enumerable:!0,value:e}),2&t&&"string"!=typeof e)for(var o in e)r.d(n,o,function(t){return e[t]}.bind(null,o));return n},r.n=function(t){var e=t&&t.__esModule?function(){return t.default}:function(){return t};return r.d(e,"a",e),e},r.o=function(t,e){return Object.prototype.hasOwnProperty.call(t,e)},r.p="",r(r.s=0)}([function(t,e,n){"use strict";var r="function"==typeof Symbol&&"symbol"==typeof Symbol.iterator?function(t){return typeof t}:function(t){return t&&"function"==typeof Symbol&&t.constructor===Symbol&&t!==Symbol.prototype?"symbol":typeof t},i=function(){function o(t,e){for(var n=0;n\u00B6'). + attr('href', '#' + this.id). + attr('title', _('Permalink to this headline')). + appendTo(this); + }); + $('dt[id]').each(function() { + $('\u00B6'). + attr('href', '#' + this.id). + attr('title', _('Permalink to this definition')). + appendTo(this); + }); + }, + + /** + * workaround a firefox stupidity + * see: https://bugzilla.mozilla.org/show_bug.cgi?id=645075 + */ + fixFirefoxAnchorBug : function() { + if (document.location.hash && $.browser.mozilla) + window.setTimeout(function() { + document.location.href += ''; + }, 10); + }, + + /** + * highlight the search words provided in the url in the text + */ + highlightSearchWords : function() { + var params = $.getQueryParameters(); + var terms = (params.highlight) ? params.highlight[0].split(/\s+/) : []; + if (terms.length) { + var body = $('div.body'); + if (!body.length) { + body = $('body'); + } + window.setTimeout(function() { + $.each(terms, function() { + body.highlightText(this.toLowerCase(), 'highlighted'); + }); + }, 10); + $('

' + _('Hide Search Matches') + '

') + .appendTo($('#searchbox')); + } + }, + + /** + * init the domain index toggle buttons + */ + initIndexTable : function() { + var togglers = $('img.toggler').click(function() { + var src = $(this).attr('src'); + var idnum = $(this).attr('id').substr(7); + $('tr.cg-' + idnum).toggle(); + if (src.substr(-9) === 'minus.png') + $(this).attr('src', src.substr(0, src.length-9) + 'plus.png'); + else + $(this).attr('src', src.substr(0, src.length-8) + 'minus.png'); + }).css('display', ''); + if (DOCUMENTATION_OPTIONS.COLLAPSE_INDEX) { + togglers.click(); + } + }, + + /** + * helper function to hide the search marks again + */ + hideSearchWords : function() { + $('#searchbox .highlight-link').fadeOut(300); + $('span.highlighted').removeClass('highlighted'); + }, + + /** + * make the url absolute + */ + makeURL : function(relativeURL) { + return DOCUMENTATION_OPTIONS.URL_ROOT + '/' + relativeURL; + }, + + /** + * get the current relative url + */ + getCurrentURL : function() { + var path = document.location.pathname; + var parts = path.split(/\//); + $.each(DOCUMENTATION_OPTIONS.URL_ROOT.split(/\//), function() { + if (this === '..') + parts.pop(); + }); + var url = parts.join('/'); + return path.substring(url.lastIndexOf('/') + 1, path.length - 1); + }, + + initOnKeyListeners: function() { + $(document).keydown(function(event) { + var activeElementType = document.activeElement.tagName; + // don't navigate when in search box or textarea + if (activeElementType !== 'TEXTAREA' && activeElementType !== 'INPUT' && activeElementType !== 'SELECT' + && !event.altKey && !event.ctrlKey && !event.metaKey && !event.shiftKey) { + switch (event.keyCode) { + case 37: // left + var prevHref = $('link[rel="prev"]').prop('href'); + if (prevHref) { + window.location.href = prevHref; + return false; + } + case 39: // right + var nextHref = $('link[rel="next"]').prop('href'); + if (nextHref) { + window.location.href = nextHref; + return false; + } + } + } + }); + } +}; + +// quick alias for translations +_ = Documentation.gettext; + +$(document).ready(function() { + Documentation.init(); +}); diff --git a/_static/documentation_options.js b/_static/documentation_options.js new file mode 100644 index 0000000..7ad534e --- /dev/null +++ b/_static/documentation_options.js @@ -0,0 +1,11 @@ +var DOCUMENTATION_OPTIONS = { + URL_ROOT: document.getElementById("documentation_options").getAttribute('data-url_root'), + VERSION: '', + LANGUAGE: 'None', + COLLAPSE_INDEX: false, + BUILDER: 'html', + FILE_SUFFIX: '.html', + HAS_SOURCE: true, + SOURCELINK_SUFFIX: '', + NAVIGATION_WITH_KEYS: true +}; \ No newline at end of file diff --git a/_static/file.png b/_static/file.png new file mode 100644 index 0000000000000000000000000000000000000000..a858a410e4faa62ce324d814e4b816fff83a6fb3 GIT binary patch literal 286 zcmV+(0pb3MP)s`hMrGg#P~ix$^RISR_I47Y|r1 z_CyJOe}D1){SET-^Amu_i71Lt6eYfZjRyw@I6OQAIXXHDfiX^GbOlHe=Ae4>0m)d(f|Me07*qoM6N<$f}vM^LjV8( literal 0 HcmV?d00001 diff --git a/_static/images/logo_binder.svg b/_static/images/logo_binder.svg new file mode 100644 index 0000000..16390ee --- /dev/null +++ b/_static/images/logo_binder.svg @@ -0,0 +1,19 @@ + + + + +logo + + + + + + + + diff --git a/_static/images/logo_colab.png b/_static/images/logo_colab.png new file mode 100644 index 0000000000000000000000000000000000000000..b7560ec216b2d1b6f77855525fe966c741833428 GIT binary patch literal 7601 zcmeI1^;ZuSFsz@@e&Hu|o~yU_Jn_7Cy4b4(M?f2S`owL6D#ysoM3Rsb4MX|l6hl52QIsX*kmQMmFZ6Xu|Wk1r15+E^+Er?@^MFpIE zq!=C|$Nn*F4aR@N|DPxS6E^f|7Z=H%T>vS)_|-RkkprWw zSGb9TlwheKfo{U5J)kX1$cHtEFe}Pa2Au|?^hCk%8gdI}l*ypIUsLXLMy9W|q-ZAw zJpZkmGRa|!=7CyrA#Bs2?5UdZ1^pDaji}+DimdE$JB@FrJvAIxy*3v#1-8OwO;OS$ zsv*P<%V4%?*Keca@o9}LMOs~ph)z!AU;${{23k&Gq7A@nDP{*I1HiTZ=Q*54?Bok) zp6L_4HhiE->YU6{m*{7O7j#SkBb9JPo!k8TD0H6{ zdSE-mmA!Js{}(?qh${0wB7Rx{*F=43D>?j3kU8MX&`sQJ+wHUD6eEr7j%*2x%5|a8 z*;AP<*tCQwj`Af5vvGHXF=9{cdzV2BMI@}VHgmol)^f>Ectcls5p3dW?40~ADd>ki za*q>v=nQQmGI5&BS!GU|iX9>qB9r=_Qm9t_Qwi+zWI zc%%oQ`P}{ZXk^}?+H!u2my^C#TD%=V|3pb$MXhJ07bx-^=oxj?ZSk!---?f2cs8_& z8?O{lvxMDZi7gsdvoZ2bmyLYs1!O1RMC)1Wv`9p-I(1pfww9siX;Lu>^>_Y=g+OHo zPm(N|h?h5Z>yze~wKtPBRv(mZx*A4R%bganw#OV=SE*=J^b#~(YfIcj(k=(i37PY7 zUiawSj8SKczPk-^=SwOOb%X+bRcFm+=N1r{{CA<=kbVq8cFGcLSGqM5FUxChbc&`o9$mUo4kZLh+%KP6m zDMd3SH~N5fH8J+8;bpxhi-9i}^PV(^u?zb49_c!Ow_!1w%w(RLEeXJoMU>Nnlc8sd z<;K$L<-WwC`NJ0PWzB59Pzbg|FZS-=xlaWDjM-PXIJ;r4qyFnFc_<-VDg5P=Zk0Pd z%f7GFg?FzC??rmjG^Ib<{cfE+dud-%)Ep=a8Q(Z-Fng}&CvD+JPdO)mL-$u4eH#LJ z7heze_GA*{rYAL;ejb#P;oTD_*Rgrw;)1(e;+zGN{)D)k?o$t&BGWEM!Hn}LQm1jd zf@B0+pEzI&qREI@Qr=#K;u~Fs)Saf>_1X|EQGz0D_a|>)d?IOck($^4a`v4Hc6sKV zgm7-VK|sz+(A$-L0BnhZ#qKk${svcv4#QmCcMCb>t9=e+^b49rrK@5C@-Qs{PN6H8Tb^nIy#)VA`)o~+c~m2m9bN}EcwI`-IP+fB&d^;19iX9{XvM6VYHE(fX{BIU zjMLmkl7p}TslG;@C!HvX=7hVy6cGIM{h7hxrM^q{j`Y4Ux1nI*k9MB?ToSK!Qpvy< zT~`Qofe|OBk8vza_r02Y;~+V6WKn(J{_?BR9@-`D&Q;nTEx7+j36Qk0(l3TahUki} z;O-FUuOnNVcc-Q3c?;A)ZpgKC-Sa8`{c}MNm$j))KPPdL#xR*0kxQz|V-;WZxI+?u zFB#~P=os0);b?+6$-z@yE%k*^!0x)K_!|4!L%ADpXqe`pG|8A+rht_!jZid=wb1j& zjPG_SeS*{ef!h*}~k!*;Aar3`tCeHO@>c{c>ak(x3f^w3+_zT>j)aP_hVoV4~^0L<5^eu_y z-@tf0YyH-(#5uTh`s3DIhpc^`UysO{L8JS|z=qnHFb)UqfMnC!Hu$=eiC+a;9t*X6R?Q8POFRq?_ak1&yP&YF6`@B=qySm8MJ)n*E zdS-&E$a$DMp!}+S%^(Q))m7O$Qece1ZtB+=H{**c0@XT53VGNeFhvnDVocubi6~ru z2X&(|kp)joFLfuG?i;d=&CZBQhez8i+lhV+c;_pEL6+Teo z1qclCF-EO~XWkH3u|unGI79@`+YLi}rF>PbBrn{PBKWF&S%K6N0u^DRx7qImnJ`+c z>Nu)TJyhpyJX_!XHh^82M+YgW&cxs(vQKEpL%}iK(hH=<@)j#E3_?a*JP@0=R z;O*(_2@>IjYLClnL+$PJ-5!vt6>UJ7$KHM3LlFFMxb19oFZ_fi@{fp};$@_n8driG z`=77&{Z^0#T>t%$hCqQi8M}0E4XipxikcsB$>o9M)rBJWQDY7UrgKAy|BP4kr`Nay z??T|Ajh_U=3lem-tL$_tEhB=Rqfi?bUj`u>$a-x5WxqHn6t4)Q-NQ^Bt-k!mcE0ES z4)*3-(5@V)=EloLT~ReorH252&Q&MWWc$oiSS{!xpO?VPpJFD-QN6c=<7HxnH1nH% zeiOM22U=%trq`HCXYNL#H!P!M1{?)QcIGYWO$;mCMHnpgd?*ZE&bmylPxndZ$B}ct zIfSCaCu!a^rBwLoo4gQJnU<%~!6cPP-qxJLZM#F&_gwU%?O$k?DIF6l%q_lvcs3})|Z?z(K3q9(BASQtZlw@+<5mv zrHuRbc}A4I9hLtxbS!@ju49VVt1XxpO?1&$LA;?ZANYo=SC^nMg{9BY`=cZcTaR{A@r{UB@;%H zPb6QWRuvU)J>>*0FB;9Uq|hH4C$u8T=T?sz{5%Ex)I%5W6wQmtel=rJ)Tbw#E7{Z;t3U zY9a$t=WkneF<9867^HBvLp>hs;A@H}9KEwn2t!?ITQ1vZ?fCFF(RfFYplQUymF`y4 z74MX)v7%4i_52G~fn=&qCfo}f%Gj8bd7dI^BDI?AlVN_!qWMJT#NBLs^p)e{tG?D4 z)|x9tIcLpO$-JtVj=#$1Y&GRE*-xUKd_{uxiZkqAudNRF!dph|+p41KtIf(8)c1p~ zv)f(_RGUK*j_{s!DNDET-@ekFNlnTXW_=+4t5>Qbq`aWl%F6e}e)<=0U{Lp}8twQ? z8cJ&^2hntuxcqQ~k;<29cTQz)@X@zbQN?f1q??MK&`gi2me&l@XLSxN|!? z;kRJcy-ahz{?{Aj;b0E9*MKf|Q@H!%2FhB8=t$dhTtR4^%hSctIRz;tXJPme_gd zLiJlhH^x9|I?_vaIKkgiAyrk&%Mv26OqK|av#t%u9aU2`wvZ61wo4$DW%z~d9P`5& zx2Zk{zL$Z1@bGicZ})KZzJKhZaZ+P!-p1uH9dgwUQ5u(q{HyTaprSe95WuIadBYv0 zPUJ~G+G2~n0DfE{7!{N*#1+?ql4nK8`Fr?o@j~3c(>T^^trK4t~7#7WQoVk)7KnFY{iPIQ?Qh8 z+Wy6Ol|m6pA8r4lQdt@$=Z{k}^_evzh~Vt_J$aBM!djok7rTfxt8f+KVv7GM1Awc>b%$6NDX zcl~`@-PYtGJSGIO(C^sr&BxXHz*cUJnB~X1`0$kX)@xH+qFRp1^Vpt^u3V$(w;_vf zHIi3Mb+A5@Nx^>r8g^tF%=j0o$Rhli22c4xiy2SEGE=Dk)m)mzF}VhHtiP43?%dTPKbDg+Gmq$pq6DlCZzY5@`})4DTSfgVh3B z6B#;izoI9B%{^V1qYVp<-KgZ=_(;UqyU^wT{IFPQ?YY4%;yq4cbgN`_dqp${t%ytU z!T>q+J?*26u4Ak4Jx#9uHgScR2!%5YX9%5Bu@HL^VaJ7%jj#ceYuaRZk7vMWX)jq| z-rX)3v33MqZ$qaWp!X$i1yJ*rOfjP-u6noa{n9pxzJw0P2+@UNLHS(-e>##A#9xc` zAr=;dh7~9d71L_&bj`DI@l$2 zSX@4j7tZbUYdo?rgctpAg3>Z@gv1{~grCRQUGVyTbzIJ-YZt2xF(cT)W0~l-76Lw* z<6YF%D4R$X>ZEj#!c)zMi018e@?^1%&N`zutD(OQ;X8am+pNW(YhRwy*%wrsnwb#T z>n{K;55wQE!cVF)X+X12fX<x`lE~DquFsMPRoBuzhuVdR8Gv zevya06i9>q3oJZyDGUHOP=iTbBg`AO7~BI0N8$lqEvK_=V)(Du!8=i|%_2^xqnCgh zYEho!c`8!%;N8>VD_@8NZxuyDHBlxl_=CBT5z4cft(NLsv9Wo81)VnjTne@sFAuLA zv^?3h>Rc?eDzkn@SvwCF^spU#ZJuQz6o4V90>Al2JL^>6N4y0wyg#4m?khQ$4$xa5 zlJZV5E$o~arUalDb_b7lXJs*(UA*P>jQ%3i`I8pyKN?*kY>iRE7J9GGiz^nA>aIV> zaJ}>Ecj_*#d8xFcjhy+6oRGfCr^qR6C2fGkhPUT-of7St?XBEaY>?_o$Y;IiV*<6d zlA;M(1^;P>tJxjiTQAB{T$TKPJ?7HfGON=ms6=%yai0?j-qHB-nhvKj_0=^YawDhO z&$wC;93X#RhmcNJTfn66z&E;UAFGeV6TsD61;r(%GZvUrDg2W3Y2hPsTqkinoI4PV zXDedcq+P^|`+Zqpt5*;9cKbAf6!xI4X{#P5OMaE4?*}B?BIY^Gyv0%UUq}lKO~C#Z zCRamrC=OeXKTKm|4p>}U!kLbE%NxPGuZ1-DR(wWFK@>24ca*qhEt5B*r|(Kty!Pj0 zZauh;NqoiV&&q9pT#S7@dl4JUVA|RmaH8kslFhypJ_)20*ebs^yXIQA(6mi|Wph<8 z=`?$6$QX%TaWE9DLjOgi>rciE+f(9`A4gn4&jZA)v29ug%2=CtvV-U|71pd@edT~> zTA~BLBxs`RYEh%@DuEBdVt=S~6x5VXGkg4=c(|;e@Uk2Mxd}~#h^+`jF}r@=C0+HS zJcg`@*AUj2Ymhzqb=;b}w_oSQ>VH<@k=B`!P>>u5;cpo7O#PB&IQ>AS{06fz5fsXyOt1R0^~JUdht$M7yYTxq$&$T&teFpg;y{BUxXR(00s6bHa2EU zQz~u3(zn7I;Ei{D%kc60jYvUAK^2vZcMr$(Mvo58z}?>{fBdZv&KdKaM(W*WeijQ+ z;}+j>_K=@gAG4KLl-oHs1uHl{4Iq_bV|(|n23Ml=$x+vE+w;rZ1-;Cgwa-{hvjGND zf$}y#wu81ZOPZ@Wj}WbIj4k%PEPTy)sLP0Kk0C=n2lpOrPl~et;FC1`zjD=4!5coL zUgdZMo&inr`+cr#<^beEmG){%LjzXvEJ;=`hMnEYG|VU#W^gR^?uh;u@MsY$78=09EY#xn`@9X5)nb~&t)6wi zB(Y#$oL!o_oI|#`LeD5m>ezV6;nKHq@ZYvUufb~M33Qw%6`GhEa}S@P!}T;dH@bLx zG_yiKDTq6zQz}25>oeWOXpL<9!kJrP)LQASx)Dh$MiaKmk}q7TZJjtiA`M6zv_)Sn zoW-S@(c2ebP+DQqvD-S;#gt=zlveyhax!aybe(eZtlKEO1+bZSMlogo_jupyterhubHub diff --git a/_static/jquery-3.4.1.js b/_static/jquery-3.4.1.js new file mode 100644 index 0000000..773ad95 --- /dev/null +++ b/_static/jquery-3.4.1.js @@ -0,0 +1,10598 @@ +/*! + * jQuery JavaScript Library v3.4.1 + * https://jquery.com/ + * + * Includes Sizzle.js + * https://sizzlejs.com/ + * + * Copyright JS Foundation and other contributors + * Released under the MIT license + * https://jquery.org/license + * + * Date: 2019-05-01T21:04Z + */ +( function( global, factory ) { + + "use strict"; + + if ( typeof module === "object" && typeof module.exports === "object" ) { + + // For CommonJS and CommonJS-like environments where a proper `window` + // is present, execute the factory and get jQuery. + // For environments that do not have a `window` with a `document` + // (such as Node.js), expose a factory as module.exports. + // This accentuates the need for the creation of a real `window`. + // e.g. var jQuery = require("jquery")(window); + // See ticket #14549 for more info. + module.exports = global.document ? + factory( global, true ) : + function( w ) { + if ( !w.document ) { + throw new Error( "jQuery requires a window with a document" ); + } + return factory( w ); + }; + } else { + factory( global ); + } + +// Pass this if window is not defined yet +} )( typeof window !== "undefined" ? window : this, function( window, noGlobal ) { + +// Edge <= 12 - 13+, Firefox <=18 - 45+, IE 10 - 11, Safari 5.1 - 9+, iOS 6 - 9.1 +// throw exceptions when non-strict code (e.g., ASP.NET 4.5) accesses strict mode +// arguments.callee.caller (trac-13335). But as of jQuery 3.0 (2016), strict mode should be common +// enough that all such attempts are guarded in a try block. +"use strict"; + +var arr = []; + +var document = window.document; + +var getProto = Object.getPrototypeOf; + +var slice = arr.slice; + +var concat = arr.concat; + +var push = arr.push; + +var indexOf = arr.indexOf; + +var class2type = {}; + +var toString = class2type.toString; + +var hasOwn = class2type.hasOwnProperty; + +var fnToString = hasOwn.toString; + +var ObjectFunctionString = fnToString.call( Object ); + +var support = {}; + +var isFunction = function isFunction( obj ) { + + // Support: Chrome <=57, Firefox <=52 + // In some browsers, typeof returns "function" for HTML elements + // (i.e., `typeof document.createElement( "object" ) === "function"`). + // We don't want to classify *any* DOM node as a function. + return typeof obj === "function" && typeof obj.nodeType !== "number"; + }; + + +var isWindow = function isWindow( obj ) { + return obj != null && obj === obj.window; + }; + + + + + var preservedScriptAttributes = { + type: true, + src: true, + nonce: true, + noModule: true + }; + + function DOMEval( code, node, doc ) { + doc = doc || document; + + var i, val, + script = doc.createElement( "script" ); + + script.text = code; + if ( node ) { + for ( i in preservedScriptAttributes ) { + + // Support: Firefox 64+, Edge 18+ + // Some browsers don't support the "nonce" property on scripts. + // On the other hand, just using `getAttribute` is not enough as + // the `nonce` attribute is reset to an empty string whenever it + // becomes browsing-context connected. + // See https://github.com/whatwg/html/issues/2369 + // See https://html.spec.whatwg.org/#nonce-attributes + // The `node.getAttribute` check was added for the sake of + // `jQuery.globalEval` so that it can fake a nonce-containing node + // via an object. + val = node[ i ] || node.getAttribute && node.getAttribute( i ); + if ( val ) { + script.setAttribute( i, val ); + } + } + } + doc.head.appendChild( script ).parentNode.removeChild( script ); + } + + +function toType( obj ) { + if ( obj == null ) { + return obj + ""; + } + + // Support: Android <=2.3 only (functionish RegExp) + return typeof obj === "object" || typeof obj === "function" ? + class2type[ toString.call( obj ) ] || "object" : + typeof obj; +} +/* global Symbol */ +// Defining this global in .eslintrc.json would create a danger of using the global +// unguarded in another place, it seems safer to define global only for this module + + + +var + version = "3.4.1", + + // Define a local copy of jQuery + jQuery = function( selector, context ) { + + // The jQuery object is actually just the init constructor 'enhanced' + // Need init if jQuery is called (just allow error to be thrown if not included) + return new jQuery.fn.init( selector, context ); + }, + + // Support: Android <=4.0 only + // Make sure we trim BOM and NBSP + rtrim = /^[\s\uFEFF\xA0]+|[\s\uFEFF\xA0]+$/g; + +jQuery.fn = jQuery.prototype = { + + // The current version of jQuery being used + jquery: version, + + constructor: jQuery, + + // The default length of a jQuery object is 0 + length: 0, + + toArray: function() { + return slice.call( this ); + }, + + // Get the Nth element in the matched element set OR + // Get the whole matched element set as a clean array + get: function( num ) { + + // Return all the elements in a clean array + if ( num == null ) { + return slice.call( this ); + } + + // Return just the one element from the set + return num < 0 ? this[ num + this.length ] : this[ num ]; + }, + + // Take an array of elements and push it onto the stack + // (returning the new matched element set) + pushStack: function( elems ) { + + // Build a new jQuery matched element set + var ret = jQuery.merge( this.constructor(), elems ); + + // Add the old object onto the stack (as a reference) + ret.prevObject = this; + + // Return the newly-formed element set + return ret; + }, + + // Execute a callback for every element in the matched set. + each: function( callback ) { + return jQuery.each( this, callback ); + }, + + map: function( callback ) { + return this.pushStack( jQuery.map( this, function( elem, i ) { + return callback.call( elem, i, elem ); + } ) ); + }, + + slice: function() { + return this.pushStack( slice.apply( this, arguments ) ); + }, + + first: function() { + return this.eq( 0 ); + }, + + last: function() { + return this.eq( -1 ); + }, + + eq: function( i ) { + var len = this.length, + j = +i + ( i < 0 ? len : 0 ); + return this.pushStack( j >= 0 && j < len ? [ this[ j ] ] : [] ); + }, + + end: function() { + return this.prevObject || this.constructor(); + }, + + // For internal use only. + // Behaves like an Array's method, not like a jQuery method. + push: push, + sort: arr.sort, + splice: arr.splice +}; + +jQuery.extend = jQuery.fn.extend = function() { + var options, name, src, copy, copyIsArray, clone, + target = arguments[ 0 ] || {}, + i = 1, + length = arguments.length, + deep = false; + + // Handle a deep copy situation + if ( typeof target === "boolean" ) { + deep = target; + + // Skip the boolean and the target + target = arguments[ i ] || {}; + i++; + } + + // Handle case when target is a string or something (possible in deep copy) + if ( typeof target !== "object" && !isFunction( target ) ) { + target = {}; + } + + // Extend jQuery itself if only one argument is passed + if ( i === length ) { + target = this; + i--; + } + + for ( ; i < length; i++ ) { + + // Only deal with non-null/undefined values + if ( ( options = arguments[ i ] ) != null ) { + + // Extend the base object + for ( name in options ) { + copy = options[ name ]; + + // Prevent Object.prototype pollution + // Prevent never-ending loop + if ( name === "__proto__" || target === copy ) { + continue; + } + + // Recurse if we're merging plain objects or arrays + if ( deep && copy && ( jQuery.isPlainObject( copy ) || + ( copyIsArray = Array.isArray( copy ) ) ) ) { + src = target[ name ]; + + // Ensure proper type for the source value + if ( copyIsArray && !Array.isArray( src ) ) { + clone = []; + } else if ( !copyIsArray && !jQuery.isPlainObject( src ) ) { + clone = {}; + } else { + clone = src; + } + copyIsArray = false; + + // Never move original objects, clone them + target[ name ] = jQuery.extend( deep, clone, copy ); + + // Don't bring in undefined values + } else if ( copy !== undefined ) { + target[ name ] = copy; + } + } + } + } + + // Return the modified object + return target; +}; + +jQuery.extend( { + + // Unique for each copy of jQuery on the page + expando: "jQuery" + ( version + Math.random() ).replace( /\D/g, "" ), + + // Assume jQuery is ready without the ready module + isReady: true, + + error: function( msg ) { + throw new Error( msg ); + }, + + noop: function() {}, + + isPlainObject: function( obj ) { + var proto, Ctor; + + // Detect obvious negatives + // Use toString instead of jQuery.type to catch host objects + if ( !obj || toString.call( obj ) !== "[object Object]" ) { + return false; + } + + proto = getProto( obj ); + + // Objects with no prototype (e.g., `Object.create( null )`) are plain + if ( !proto ) { + return true; + } + + // Objects with prototype are plain iff they were constructed by a global Object function + Ctor = hasOwn.call( proto, "constructor" ) && proto.constructor; + return typeof Ctor === "function" && fnToString.call( Ctor ) === ObjectFunctionString; + }, + + isEmptyObject: function( obj ) { + var name; + + for ( name in obj ) { + return false; + } + return true; + }, + + // Evaluates a script in a global context + globalEval: function( code, options ) { + DOMEval( code, { nonce: options && options.nonce } ); + }, + + each: function( obj, callback ) { + var length, i = 0; + + if ( isArrayLike( obj ) ) { + length = obj.length; + for ( ; i < length; i++ ) { + if ( callback.call( obj[ i ], i, obj[ i ] ) === false ) { + break; + } + } + } else { + for ( i in obj ) { + if ( callback.call( obj[ i ], i, obj[ i ] ) === false ) { + break; + } + } + } + + return obj; + }, + + // Support: Android <=4.0 only + trim: function( text ) { + return text == null ? + "" : + ( text + "" ).replace( rtrim, "" ); + }, + + // results is for internal usage only + makeArray: function( arr, results ) { + var ret = results || []; + + if ( arr != null ) { + if ( isArrayLike( Object( arr ) ) ) { + jQuery.merge( ret, + typeof arr === "string" ? + [ arr ] : arr + ); + } else { + push.call( ret, arr ); + } + } + + return ret; + }, + + inArray: function( elem, arr, i ) { + return arr == null ? -1 : indexOf.call( arr, elem, i ); + }, + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + merge: function( first, second ) { + var len = +second.length, + j = 0, + i = first.length; + + for ( ; j < len; j++ ) { + first[ i++ ] = second[ j ]; + } + + first.length = i; + + return first; + }, + + grep: function( elems, callback, invert ) { + var callbackInverse, + matches = [], + i = 0, + length = elems.length, + callbackExpect = !invert; + + // Go through the array, only saving the items + // that pass the validator function + for ( ; i < length; i++ ) { + callbackInverse = !callback( elems[ i ], i ); + if ( callbackInverse !== callbackExpect ) { + matches.push( elems[ i ] ); + } + } + + return matches; + }, + + // arg is for internal usage only + map: function( elems, callback, arg ) { + var length, value, + i = 0, + ret = []; + + // Go through the array, translating each of the items to their new values + if ( isArrayLike( elems ) ) { + length = elems.length; + for ( ; i < length; i++ ) { + value = callback( elems[ i ], i, arg ); + + if ( value != null ) { + ret.push( value ); + } + } + + // Go through every key on the object, + } else { + for ( i in elems ) { + value = callback( elems[ i ], i, arg ); + + if ( value != null ) { + ret.push( value ); + } + } + } + + // Flatten any nested arrays + return concat.apply( [], ret ); + }, + + // A global GUID counter for objects + guid: 1, + + // jQuery.support is not used in Core but other projects attach their + // properties to it so it needs to exist. + support: support +} ); + +if ( typeof Symbol === "function" ) { + jQuery.fn[ Symbol.iterator ] = arr[ Symbol.iterator ]; +} + +// Populate the class2type map +jQuery.each( "Boolean Number String Function Array Date RegExp Object Error Symbol".split( " " ), +function( i, name ) { + class2type[ "[object " + name + "]" ] = name.toLowerCase(); +} ); + +function isArrayLike( obj ) { + + // Support: real iOS 8.2 only (not reproducible in simulator) + // `in` check used to prevent JIT error (gh-2145) + // hasOwn isn't used here due to false negatives + // regarding Nodelist length in IE + var length = !!obj && "length" in obj && obj.length, + type = toType( obj ); + + if ( isFunction( obj ) || isWindow( obj ) ) { + return false; + } + + return type === "array" || length === 0 || + typeof length === "number" && length > 0 && ( length - 1 ) in obj; +} +var Sizzle = +/*! + * Sizzle CSS Selector Engine v2.3.4 + * https://sizzlejs.com/ + * + * Copyright JS Foundation and other contributors + * Released under the MIT license + * https://js.foundation/ + * + * Date: 2019-04-08 + */ +(function( window ) { + +var i, + support, + Expr, + getText, + isXML, + tokenize, + compile, + select, + outermostContext, + sortInput, + hasDuplicate, + + // Local document vars + setDocument, + document, + docElem, + documentIsHTML, + rbuggyQSA, + rbuggyMatches, + matches, + contains, + + // Instance-specific data + expando = "sizzle" + 1 * new Date(), + preferredDoc = window.document, + dirruns = 0, + done = 0, + classCache = createCache(), + tokenCache = createCache(), + compilerCache = createCache(), + nonnativeSelectorCache = createCache(), + sortOrder = function( a, b ) { + if ( a === b ) { + hasDuplicate = true; + } + return 0; + }, + + // Instance methods + hasOwn = ({}).hasOwnProperty, + arr = [], + pop = arr.pop, + push_native = arr.push, + push = arr.push, + slice = arr.slice, + // Use a stripped-down indexOf as it's faster than native + // https://jsperf.com/thor-indexof-vs-for/5 + indexOf = function( list, elem ) { + var i = 0, + len = list.length; + for ( ; i < len; i++ ) { + if ( list[i] === elem ) { + return i; + } + } + return -1; + }, + + booleans = "checked|selected|async|autofocus|autoplay|controls|defer|disabled|hidden|ismap|loop|multiple|open|readonly|required|scoped", + + // Regular expressions + + // http://www.w3.org/TR/css3-selectors/#whitespace + whitespace = "[\\x20\\t\\r\\n\\f]", + + // http://www.w3.org/TR/CSS21/syndata.html#value-def-identifier + identifier = "(?:\\\\.|[\\w-]|[^\0-\\xa0])+", + + // Attribute selectors: http://www.w3.org/TR/selectors/#attribute-selectors + attributes = "\\[" + whitespace + "*(" + identifier + ")(?:" + whitespace + + // Operator (capture 2) + "*([*^$|!~]?=)" + whitespace + + // "Attribute values must be CSS identifiers [capture 5] or strings [capture 3 or capture 4]" + "*(?:'((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\"|(" + identifier + "))|)" + whitespace + + "*\\]", + + pseudos = ":(" + identifier + ")(?:\\((" + + // To reduce the number of selectors needing tokenize in the preFilter, prefer arguments: + // 1. quoted (capture 3; capture 4 or capture 5) + "('((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\")|" + + // 2. simple (capture 6) + "((?:\\\\.|[^\\\\()[\\]]|" + attributes + ")*)|" + + // 3. anything else (capture 2) + ".*" + + ")\\)|)", + + // Leading and non-escaped trailing whitespace, capturing some non-whitespace characters preceding the latter + rwhitespace = new RegExp( whitespace + "+", "g" ), + rtrim = new RegExp( "^" + whitespace + "+|((?:^|[^\\\\])(?:\\\\.)*)" + whitespace + "+$", "g" ), + + rcomma = new RegExp( "^" + whitespace + "*," + whitespace + "*" ), + rcombinators = new RegExp( "^" + whitespace + "*([>+~]|" + whitespace + ")" + whitespace + "*" ), + rdescend = new RegExp( whitespace + "|>" ), + + rpseudo = new RegExp( pseudos ), + ridentifier = new RegExp( "^" + identifier + "$" ), + + matchExpr = { + "ID": new RegExp( "^#(" + identifier + ")" ), + "CLASS": new RegExp( "^\\.(" + identifier + ")" ), + "TAG": new RegExp( "^(" + identifier + "|[*])" ), + "ATTR": new RegExp( "^" + attributes ), + "PSEUDO": new RegExp( "^" + pseudos ), + "CHILD": new RegExp( "^:(only|first|last|nth|nth-last)-(child|of-type)(?:\\(" + whitespace + + "*(even|odd|(([+-]|)(\\d*)n|)" + whitespace + "*(?:([+-]|)" + whitespace + + "*(\\d+)|))" + whitespace + "*\\)|)", "i" ), + "bool": new RegExp( "^(?:" + booleans + ")$", "i" ), + // For use in libraries implementing .is() + // We use this for POS matching in `select` + "needsContext": new RegExp( "^" + whitespace + "*[>+~]|:(even|odd|eq|gt|lt|nth|first|last)(?:\\(" + + whitespace + "*((?:-\\d)?\\d*)" + whitespace + "*\\)|)(?=[^-]|$)", "i" ) + }, + + rhtml = /HTML$/i, + rinputs = /^(?:input|select|textarea|button)$/i, + rheader = /^h\d$/i, + + rnative = /^[^{]+\{\s*\[native \w/, + + // Easily-parseable/retrievable ID or TAG or CLASS selectors + rquickExpr = /^(?:#([\w-]+)|(\w+)|\.([\w-]+))$/, + + rsibling = /[+~]/, + + // CSS escapes + // http://www.w3.org/TR/CSS21/syndata.html#escaped-characters + runescape = new RegExp( "\\\\([\\da-f]{1,6}" + whitespace + "?|(" + whitespace + ")|.)", "ig" ), + funescape = function( _, escaped, escapedWhitespace ) { + var high = "0x" + escaped - 0x10000; + // NaN means non-codepoint + // Support: Firefox<24 + // Workaround erroneous numeric interpretation of +"0x" + return high !== high || escapedWhitespace ? + escaped : + high < 0 ? + // BMP codepoint + String.fromCharCode( high + 0x10000 ) : + // Supplemental Plane codepoint (surrogate pair) + String.fromCharCode( high >> 10 | 0xD800, high & 0x3FF | 0xDC00 ); + }, + + // CSS string/identifier serialization + // https://drafts.csswg.org/cssom/#common-serializing-idioms + rcssescape = /([\0-\x1f\x7f]|^-?\d)|^-$|[^\0-\x1f\x7f-\uFFFF\w-]/g, + fcssescape = function( ch, asCodePoint ) { + if ( asCodePoint ) { + + // U+0000 NULL becomes U+FFFD REPLACEMENT CHARACTER + if ( ch === "\0" ) { + return "\uFFFD"; + } + + // Control characters and (dependent upon position) numbers get escaped as code points + return ch.slice( 0, -1 ) + "\\" + ch.charCodeAt( ch.length - 1 ).toString( 16 ) + " "; + } + + // Other potentially-special ASCII characters get backslash-escaped + return "\\" + ch; + }, + + // Used for iframes + // See setDocument() + // Removing the function wrapper causes a "Permission Denied" + // error in IE + unloadHandler = function() { + setDocument(); + }, + + inDisabledFieldset = addCombinator( + function( elem ) { + return elem.disabled === true && elem.nodeName.toLowerCase() === "fieldset"; + }, + { dir: "parentNode", next: "legend" } + ); + +// Optimize for push.apply( _, NodeList ) +try { + push.apply( + (arr = slice.call( preferredDoc.childNodes )), + preferredDoc.childNodes + ); + // Support: Android<4.0 + // Detect silently failing push.apply + arr[ preferredDoc.childNodes.length ].nodeType; +} catch ( e ) { + push = { apply: arr.length ? + + // Leverage slice if possible + function( target, els ) { + push_native.apply( target, slice.call(els) ); + } : + + // Support: IE<9 + // Otherwise append directly + function( target, els ) { + var j = target.length, + i = 0; + // Can't trust NodeList.length + while ( (target[j++] = els[i++]) ) {} + target.length = j - 1; + } + }; +} + +function Sizzle( selector, context, results, seed ) { + var m, i, elem, nid, match, groups, newSelector, + newContext = context && context.ownerDocument, + + // nodeType defaults to 9, since context defaults to document + nodeType = context ? context.nodeType : 9; + + results = results || []; + + // Return early from calls with invalid selector or context + if ( typeof selector !== "string" || !selector || + nodeType !== 1 && nodeType !== 9 && nodeType !== 11 ) { + + return results; + } + + // Try to shortcut find operations (as opposed to filters) in HTML documents + if ( !seed ) { + + if ( ( context ? context.ownerDocument || context : preferredDoc ) !== document ) { + setDocument( context ); + } + context = context || document; + + if ( documentIsHTML ) { + + // If the selector is sufficiently simple, try using a "get*By*" DOM method + // (excepting DocumentFragment context, where the methods don't exist) + if ( nodeType !== 11 && (match = rquickExpr.exec( selector )) ) { + + // ID selector + if ( (m = match[1]) ) { + + // Document context + if ( nodeType === 9 ) { + if ( (elem = context.getElementById( m )) ) { + + // Support: IE, Opera, Webkit + // TODO: identify versions + // getElementById can match elements by name instead of ID + if ( elem.id === m ) { + results.push( elem ); + return results; + } + } else { + return results; + } + + // Element context + } else { + + // Support: IE, Opera, Webkit + // TODO: identify versions + // getElementById can match elements by name instead of ID + if ( newContext && (elem = newContext.getElementById( m )) && + contains( context, elem ) && + elem.id === m ) { + + results.push( elem ); + return results; + } + } + + // Type selector + } else if ( match[2] ) { + push.apply( results, context.getElementsByTagName( selector ) ); + return results; + + // Class selector + } else if ( (m = match[3]) && support.getElementsByClassName && + context.getElementsByClassName ) { + + push.apply( results, context.getElementsByClassName( m ) ); + return results; + } + } + + // Take advantage of querySelectorAll + if ( support.qsa && + !nonnativeSelectorCache[ selector + " " ] && + (!rbuggyQSA || !rbuggyQSA.test( selector )) && + + // Support: IE 8 only + // Exclude object elements + (nodeType !== 1 || context.nodeName.toLowerCase() !== "object") ) { + + newSelector = selector; + newContext = context; + + // qSA considers elements outside a scoping root when evaluating child or + // descendant combinators, which is not what we want. + // In such cases, we work around the behavior by prefixing every selector in the + // list with an ID selector referencing the scope context. + // Thanks to Andrew Dupont for this technique. + if ( nodeType === 1 && rdescend.test( selector ) ) { + + // Capture the context ID, setting it first if necessary + if ( (nid = context.getAttribute( "id" )) ) { + nid = nid.replace( rcssescape, fcssescape ); + } else { + context.setAttribute( "id", (nid = expando) ); + } + + // Prefix every selector in the list + groups = tokenize( selector ); + i = groups.length; + while ( i-- ) { + groups[i] = "#" + nid + " " + toSelector( groups[i] ); + } + newSelector = groups.join( "," ); + + // Expand context for sibling selectors + newContext = rsibling.test( selector ) && testContext( context.parentNode ) || + context; + } + + try { + push.apply( results, + newContext.querySelectorAll( newSelector ) + ); + return results; + } catch ( qsaError ) { + nonnativeSelectorCache( selector, true ); + } finally { + if ( nid === expando ) { + context.removeAttribute( "id" ); + } + } + } + } + } + + // All others + return select( selector.replace( rtrim, "$1" ), context, results, seed ); +} + +/** + * Create key-value caches of limited size + * @returns {function(string, object)} Returns the Object data after storing it on itself with + * property name the (space-suffixed) string and (if the cache is larger than Expr.cacheLength) + * deleting the oldest entry + */ +function createCache() { + var keys = []; + + function cache( key, value ) { + // Use (key + " ") to avoid collision with native prototype properties (see Issue #157) + if ( keys.push( key + " " ) > Expr.cacheLength ) { + // Only keep the most recent entries + delete cache[ keys.shift() ]; + } + return (cache[ key + " " ] = value); + } + return cache; +} + +/** + * Mark a function for special use by Sizzle + * @param {Function} fn The function to mark + */ +function markFunction( fn ) { + fn[ expando ] = true; + return fn; +} + +/** + * Support testing using an element + * @param {Function} fn Passed the created element and returns a boolean result + */ +function assert( fn ) { + var el = document.createElement("fieldset"); + + try { + return !!fn( el ); + } catch (e) { + return false; + } finally { + // Remove from its parent by default + if ( el.parentNode ) { + el.parentNode.removeChild( el ); + } + // release memory in IE + el = null; + } +} + +/** + * Adds the same handler for all of the specified attrs + * @param {String} attrs Pipe-separated list of attributes + * @param {Function} handler The method that will be applied + */ +function addHandle( attrs, handler ) { + var arr = attrs.split("|"), + i = arr.length; + + while ( i-- ) { + Expr.attrHandle[ arr[i] ] = handler; + } +} + +/** + * Checks document order of two siblings + * @param {Element} a + * @param {Element} b + * @returns {Number} Returns less than 0 if a precedes b, greater than 0 if a follows b + */ +function siblingCheck( a, b ) { + var cur = b && a, + diff = cur && a.nodeType === 1 && b.nodeType === 1 && + a.sourceIndex - b.sourceIndex; + + // Use IE sourceIndex if available on both nodes + if ( diff ) { + return diff; + } + + // Check if b follows a + if ( cur ) { + while ( (cur = cur.nextSibling) ) { + if ( cur === b ) { + return -1; + } + } + } + + return a ? 1 : -1; +} + +/** + * Returns a function to use in pseudos for input types + * @param {String} type + */ +function createInputPseudo( type ) { + return function( elem ) { + var name = elem.nodeName.toLowerCase(); + return name === "input" && elem.type === type; + }; +} + +/** + * Returns a function to use in pseudos for buttons + * @param {String} type + */ +function createButtonPseudo( type ) { + return function( elem ) { + var name = elem.nodeName.toLowerCase(); + return (name === "input" || name === "button") && elem.type === type; + }; +} + +/** + * Returns a function to use in pseudos for :enabled/:disabled + * @param {Boolean} disabled true for :disabled; false for :enabled + */ +function createDisabledPseudo( disabled ) { + + // Known :disabled false positives: fieldset[disabled] > legend:nth-of-type(n+2) :can-disable + return function( elem ) { + + // Only certain elements can match :enabled or :disabled + // https://html.spec.whatwg.org/multipage/scripting.html#selector-enabled + // https://html.spec.whatwg.org/multipage/scripting.html#selector-disabled + if ( "form" in elem ) { + + // Check for inherited disabledness on relevant non-disabled elements: + // * listed form-associated elements in a disabled fieldset + // https://html.spec.whatwg.org/multipage/forms.html#category-listed + // https://html.spec.whatwg.org/multipage/forms.html#concept-fe-disabled + // * option elements in a disabled optgroup + // https://html.spec.whatwg.org/multipage/forms.html#concept-option-disabled + // All such elements have a "form" property. + if ( elem.parentNode && elem.disabled === false ) { + + // Option elements defer to a parent optgroup if present + if ( "label" in elem ) { + if ( "label" in elem.parentNode ) { + return elem.parentNode.disabled === disabled; + } else { + return elem.disabled === disabled; + } + } + + // Support: IE 6 - 11 + // Use the isDisabled shortcut property to check for disabled fieldset ancestors + return elem.isDisabled === disabled || + + // Where there is no isDisabled, check manually + /* jshint -W018 */ + elem.isDisabled !== !disabled && + inDisabledFieldset( elem ) === disabled; + } + + return elem.disabled === disabled; + + // Try to winnow out elements that can't be disabled before trusting the disabled property. + // Some victims get caught in our net (label, legend, menu, track), but it shouldn't + // even exist on them, let alone have a boolean value. + } else if ( "label" in elem ) { + return elem.disabled === disabled; + } + + // Remaining elements are neither :enabled nor :disabled + return false; + }; +} + +/** + * Returns a function to use in pseudos for positionals + * @param {Function} fn + */ +function createPositionalPseudo( fn ) { + return markFunction(function( argument ) { + argument = +argument; + return markFunction(function( seed, matches ) { + var j, + matchIndexes = fn( [], seed.length, argument ), + i = matchIndexes.length; + + // Match elements found at the specified indexes + while ( i-- ) { + if ( seed[ (j = matchIndexes[i]) ] ) { + seed[j] = !(matches[j] = seed[j]); + } + } + }); + }); +} + +/** + * Checks a node for validity as a Sizzle context + * @param {Element|Object=} context + * @returns {Element|Object|Boolean} The input node if acceptable, otherwise a falsy value + */ +function testContext( context ) { + return context && typeof context.getElementsByTagName !== "undefined" && context; +} + +// Expose support vars for convenience +support = Sizzle.support = {}; + +/** + * Detects XML nodes + * @param {Element|Object} elem An element or a document + * @returns {Boolean} True iff elem is a non-HTML XML node + */ +isXML = Sizzle.isXML = function( elem ) { + var namespace = elem.namespaceURI, + docElem = (elem.ownerDocument || elem).documentElement; + + // Support: IE <=8 + // Assume HTML when documentElement doesn't yet exist, such as inside loading iframes + // https://bugs.jquery.com/ticket/4833 + return !rhtml.test( namespace || docElem && docElem.nodeName || "HTML" ); +}; + +/** + * Sets document-related variables once based on the current document + * @param {Element|Object} [doc] An element or document object to use to set the document + * @returns {Object} Returns the current document + */ +setDocument = Sizzle.setDocument = function( node ) { + var hasCompare, subWindow, + doc = node ? node.ownerDocument || node : preferredDoc; + + // Return early if doc is invalid or already selected + if ( doc === document || doc.nodeType !== 9 || !doc.documentElement ) { + return document; + } + + // Update global variables + document = doc; + docElem = document.documentElement; + documentIsHTML = !isXML( document ); + + // Support: IE 9-11, Edge + // Accessing iframe documents after unload throws "permission denied" errors (jQuery #13936) + if ( preferredDoc !== document && + (subWindow = document.defaultView) && subWindow.top !== subWindow ) { + + // Support: IE 11, Edge + if ( subWindow.addEventListener ) { + subWindow.addEventListener( "unload", unloadHandler, false ); + + // Support: IE 9 - 10 only + } else if ( subWindow.attachEvent ) { + subWindow.attachEvent( "onunload", unloadHandler ); + } + } + + /* Attributes + ---------------------------------------------------------------------- */ + + // Support: IE<8 + // Verify that getAttribute really returns attributes and not properties + // (excepting IE8 booleans) + support.attributes = assert(function( el ) { + el.className = "i"; + return !el.getAttribute("className"); + }); + + /* getElement(s)By* + ---------------------------------------------------------------------- */ + + // Check if getElementsByTagName("*") returns only elements + support.getElementsByTagName = assert(function( el ) { + el.appendChild( document.createComment("") ); + return !el.getElementsByTagName("*").length; + }); + + // Support: IE<9 + support.getElementsByClassName = rnative.test( document.getElementsByClassName ); + + // Support: IE<10 + // Check if getElementById returns elements by name + // The broken getElementById methods don't pick up programmatically-set names, + // so use a roundabout getElementsByName test + support.getById = assert(function( el ) { + docElem.appendChild( el ).id = expando; + return !document.getElementsByName || !document.getElementsByName( expando ).length; + }); + + // ID filter and find + if ( support.getById ) { + Expr.filter["ID"] = function( id ) { + var attrId = id.replace( runescape, funescape ); + return function( elem ) { + return elem.getAttribute("id") === attrId; + }; + }; + Expr.find["ID"] = function( id, context ) { + if ( typeof context.getElementById !== "undefined" && documentIsHTML ) { + var elem = context.getElementById( id ); + return elem ? [ elem ] : []; + } + }; + } else { + Expr.filter["ID"] = function( id ) { + var attrId = id.replace( runescape, funescape ); + return function( elem ) { + var node = typeof elem.getAttributeNode !== "undefined" && + elem.getAttributeNode("id"); + return node && node.value === attrId; + }; + }; + + // Support: IE 6 - 7 only + // getElementById is not reliable as a find shortcut + Expr.find["ID"] = function( id, context ) { + if ( typeof context.getElementById !== "undefined" && documentIsHTML ) { + var node, i, elems, + elem = context.getElementById( id ); + + if ( elem ) { + + // Verify the id attribute + node = elem.getAttributeNode("id"); + if ( node && node.value === id ) { + return [ elem ]; + } + + // Fall back on getElementsByName + elems = context.getElementsByName( id ); + i = 0; + while ( (elem = elems[i++]) ) { + node = elem.getAttributeNode("id"); + if ( node && node.value === id ) { + return [ elem ]; + } + } + } + + return []; + } + }; + } + + // Tag + Expr.find["TAG"] = support.getElementsByTagName ? + function( tag, context ) { + if ( typeof context.getElementsByTagName !== "undefined" ) { + return context.getElementsByTagName( tag ); + + // DocumentFragment nodes don't have gEBTN + } else if ( support.qsa ) { + return context.querySelectorAll( tag ); + } + } : + + function( tag, context ) { + var elem, + tmp = [], + i = 0, + // By happy coincidence, a (broken) gEBTN appears on DocumentFragment nodes too + results = context.getElementsByTagName( tag ); + + // Filter out possible comments + if ( tag === "*" ) { + while ( (elem = results[i++]) ) { + if ( elem.nodeType === 1 ) { + tmp.push( elem ); + } + } + + return tmp; + } + return results; + }; + + // Class + Expr.find["CLASS"] = support.getElementsByClassName && function( className, context ) { + if ( typeof context.getElementsByClassName !== "undefined" && documentIsHTML ) { + return context.getElementsByClassName( className ); + } + }; + + /* QSA/matchesSelector + ---------------------------------------------------------------------- */ + + // QSA and matchesSelector support + + // matchesSelector(:active) reports false when true (IE9/Opera 11.5) + rbuggyMatches = []; + + // qSa(:focus) reports false when true (Chrome 21) + // We allow this because of a bug in IE8/9 that throws an error + // whenever `document.activeElement` is accessed on an iframe + // So, we allow :focus to pass through QSA all the time to avoid the IE error + // See https://bugs.jquery.com/ticket/13378 + rbuggyQSA = []; + + if ( (support.qsa = rnative.test( document.querySelectorAll )) ) { + // Build QSA regex + // Regex strategy adopted from Diego Perini + assert(function( el ) { + // Select is set to empty string on purpose + // This is to test IE's treatment of not explicitly + // setting a boolean content attribute, + // since its presence should be enough + // https://bugs.jquery.com/ticket/12359 + docElem.appendChild( el ).innerHTML = "" + + ""; + + // Support: IE8, Opera 11-12.16 + // Nothing should be selected when empty strings follow ^= or $= or *= + // The test attribute must be unknown in Opera but "safe" for WinRT + // https://msdn.microsoft.com/en-us/library/ie/hh465388.aspx#attribute_section + if ( el.querySelectorAll("[msallowcapture^='']").length ) { + rbuggyQSA.push( "[*^$]=" + whitespace + "*(?:''|\"\")" ); + } + + // Support: IE8 + // Boolean attributes and "value" are not treated correctly + if ( !el.querySelectorAll("[selected]").length ) { + rbuggyQSA.push( "\\[" + whitespace + "*(?:value|" + booleans + ")" ); + } + + // Support: Chrome<29, Android<4.4, Safari<7.0+, iOS<7.0+, PhantomJS<1.9.8+ + if ( !el.querySelectorAll( "[id~=" + expando + "-]" ).length ) { + rbuggyQSA.push("~="); + } + + // Webkit/Opera - :checked should return selected option elements + // http://www.w3.org/TR/2011/REC-css3-selectors-20110929/#checked + // IE8 throws error here and will not see later tests + if ( !el.querySelectorAll(":checked").length ) { + rbuggyQSA.push(":checked"); + } + + // Support: Safari 8+, iOS 8+ + // https://bugs.webkit.org/show_bug.cgi?id=136851 + // In-page `selector#id sibling-combinator selector` fails + if ( !el.querySelectorAll( "a#" + expando + "+*" ).length ) { + rbuggyQSA.push(".#.+[+~]"); + } + }); + + assert(function( el ) { + el.innerHTML = "" + + ""; + + // Support: Windows 8 Native Apps + // The type and name attributes are restricted during .innerHTML assignment + var input = document.createElement("input"); + input.setAttribute( "type", "hidden" ); + el.appendChild( input ).setAttribute( "name", "D" ); + + // Support: IE8 + // Enforce case-sensitivity of name attribute + if ( el.querySelectorAll("[name=d]").length ) { + rbuggyQSA.push( "name" + whitespace + "*[*^$|!~]?=" ); + } + + // FF 3.5 - :enabled/:disabled and hidden elements (hidden elements are still enabled) + // IE8 throws error here and will not see later tests + if ( el.querySelectorAll(":enabled").length !== 2 ) { + rbuggyQSA.push( ":enabled", ":disabled" ); + } + + // Support: IE9-11+ + // IE's :disabled selector does not pick up the children of disabled fieldsets + docElem.appendChild( el ).disabled = true; + if ( el.querySelectorAll(":disabled").length !== 2 ) { + rbuggyQSA.push( ":enabled", ":disabled" ); + } + + // Opera 10-11 does not throw on post-comma invalid pseudos + el.querySelectorAll("*,:x"); + rbuggyQSA.push(",.*:"); + }); + } + + if ( (support.matchesSelector = rnative.test( (matches = docElem.matches || + docElem.webkitMatchesSelector || + docElem.mozMatchesSelector || + docElem.oMatchesSelector || + docElem.msMatchesSelector) )) ) { + + assert(function( el ) { + // Check to see if it's possible to do matchesSelector + // on a disconnected node (IE 9) + support.disconnectedMatch = matches.call( el, "*" ); + + // This should fail with an exception + // Gecko does not error, returns false instead + matches.call( el, "[s!='']:x" ); + rbuggyMatches.push( "!=", pseudos ); + }); + } + + rbuggyQSA = rbuggyQSA.length && new RegExp( rbuggyQSA.join("|") ); + rbuggyMatches = rbuggyMatches.length && new RegExp( rbuggyMatches.join("|") ); + + /* Contains + ---------------------------------------------------------------------- */ + hasCompare = rnative.test( docElem.compareDocumentPosition ); + + // Element contains another + // Purposefully self-exclusive + // As in, an element does not contain itself + contains = hasCompare || rnative.test( docElem.contains ) ? + function( a, b ) { + var adown = a.nodeType === 9 ? a.documentElement : a, + bup = b && b.parentNode; + return a === bup || !!( bup && bup.nodeType === 1 && ( + adown.contains ? + adown.contains( bup ) : + a.compareDocumentPosition && a.compareDocumentPosition( bup ) & 16 + )); + } : + function( a, b ) { + if ( b ) { + while ( (b = b.parentNode) ) { + if ( b === a ) { + return true; + } + } + } + return false; + }; + + /* Sorting + ---------------------------------------------------------------------- */ + + // Document order sorting + sortOrder = hasCompare ? + function( a, b ) { + + // Flag for duplicate removal + if ( a === b ) { + hasDuplicate = true; + return 0; + } + + // Sort on method existence if only one input has compareDocumentPosition + var compare = !a.compareDocumentPosition - !b.compareDocumentPosition; + if ( compare ) { + return compare; + } + + // Calculate position if both inputs belong to the same document + compare = ( a.ownerDocument || a ) === ( b.ownerDocument || b ) ? + a.compareDocumentPosition( b ) : + + // Otherwise we know they are disconnected + 1; + + // Disconnected nodes + if ( compare & 1 || + (!support.sortDetached && b.compareDocumentPosition( a ) === compare) ) { + + // Choose the first element that is related to our preferred document + if ( a === document || a.ownerDocument === preferredDoc && contains(preferredDoc, a) ) { + return -1; + } + if ( b === document || b.ownerDocument === preferredDoc && contains(preferredDoc, b) ) { + return 1; + } + + // Maintain original order + return sortInput ? + ( indexOf( sortInput, a ) - indexOf( sortInput, b ) ) : + 0; + } + + return compare & 4 ? -1 : 1; + } : + function( a, b ) { + // Exit early if the nodes are identical + if ( a === b ) { + hasDuplicate = true; + return 0; + } + + var cur, + i = 0, + aup = a.parentNode, + bup = b.parentNode, + ap = [ a ], + bp = [ b ]; + + // Parentless nodes are either documents or disconnected + if ( !aup || !bup ) { + return a === document ? -1 : + b === document ? 1 : + aup ? -1 : + bup ? 1 : + sortInput ? + ( indexOf( sortInput, a ) - indexOf( sortInput, b ) ) : + 0; + + // If the nodes are siblings, we can do a quick check + } else if ( aup === bup ) { + return siblingCheck( a, b ); + } + + // Otherwise we need full lists of their ancestors for comparison + cur = a; + while ( (cur = cur.parentNode) ) { + ap.unshift( cur ); + } + cur = b; + while ( (cur = cur.parentNode) ) { + bp.unshift( cur ); + } + + // Walk down the tree looking for a discrepancy + while ( ap[i] === bp[i] ) { + i++; + } + + return i ? + // Do a sibling check if the nodes have a common ancestor + siblingCheck( ap[i], bp[i] ) : + + // Otherwise nodes in our document sort first + ap[i] === preferredDoc ? -1 : + bp[i] === preferredDoc ? 1 : + 0; + }; + + return document; +}; + +Sizzle.matches = function( expr, elements ) { + return Sizzle( expr, null, null, elements ); +}; + +Sizzle.matchesSelector = function( elem, expr ) { + // Set document vars if needed + if ( ( elem.ownerDocument || elem ) !== document ) { + setDocument( elem ); + } + + if ( support.matchesSelector && documentIsHTML && + !nonnativeSelectorCache[ expr + " " ] && + ( !rbuggyMatches || !rbuggyMatches.test( expr ) ) && + ( !rbuggyQSA || !rbuggyQSA.test( expr ) ) ) { + + try { + var ret = matches.call( elem, expr ); + + // IE 9's matchesSelector returns false on disconnected nodes + if ( ret || support.disconnectedMatch || + // As well, disconnected nodes are said to be in a document + // fragment in IE 9 + elem.document && elem.document.nodeType !== 11 ) { + return ret; + } + } catch (e) { + nonnativeSelectorCache( expr, true ); + } + } + + return Sizzle( expr, document, null, [ elem ] ).length > 0; +}; + +Sizzle.contains = function( context, elem ) { + // Set document vars if needed + if ( ( context.ownerDocument || context ) !== document ) { + setDocument( context ); + } + return contains( context, elem ); +}; + +Sizzle.attr = function( elem, name ) { + // Set document vars if needed + if ( ( elem.ownerDocument || elem ) !== document ) { + setDocument( elem ); + } + + var fn = Expr.attrHandle[ name.toLowerCase() ], + // Don't get fooled by Object.prototype properties (jQuery #13807) + val = fn && hasOwn.call( Expr.attrHandle, name.toLowerCase() ) ? + fn( elem, name, !documentIsHTML ) : + undefined; + + return val !== undefined ? + val : + support.attributes || !documentIsHTML ? + elem.getAttribute( name ) : + (val = elem.getAttributeNode(name)) && val.specified ? + val.value : + null; +}; + +Sizzle.escape = function( sel ) { + return (sel + "").replace( rcssescape, fcssescape ); +}; + +Sizzle.error = function( msg ) { + throw new Error( "Syntax error, unrecognized expression: " + msg ); +}; + +/** + * Document sorting and removing duplicates + * @param {ArrayLike} results + */ +Sizzle.uniqueSort = function( results ) { + var elem, + duplicates = [], + j = 0, + i = 0; + + // Unless we *know* we can detect duplicates, assume their presence + hasDuplicate = !support.detectDuplicates; + sortInput = !support.sortStable && results.slice( 0 ); + results.sort( sortOrder ); + + if ( hasDuplicate ) { + while ( (elem = results[i++]) ) { + if ( elem === results[ i ] ) { + j = duplicates.push( i ); + } + } + while ( j-- ) { + results.splice( duplicates[ j ], 1 ); + } + } + + // Clear input after sorting to release objects + // See https://github.com/jquery/sizzle/pull/225 + sortInput = null; + + return results; +}; + +/** + * Utility function for retrieving the text value of an array of DOM nodes + * @param {Array|Element} elem + */ +getText = Sizzle.getText = function( elem ) { + var node, + ret = "", + i = 0, + nodeType = elem.nodeType; + + if ( !nodeType ) { + // If no nodeType, this is expected to be an array + while ( (node = elem[i++]) ) { + // Do not traverse comment nodes + ret += getText( node ); + } + } else if ( nodeType === 1 || nodeType === 9 || nodeType === 11 ) { + // Use textContent for elements + // innerText usage removed for consistency of new lines (jQuery #11153) + if ( typeof elem.textContent === "string" ) { + return elem.textContent; + } else { + // Traverse its children + for ( elem = elem.firstChild; elem; elem = elem.nextSibling ) { + ret += getText( elem ); + } + } + } else if ( nodeType === 3 || nodeType === 4 ) { + return elem.nodeValue; + } + // Do not include comment or processing instruction nodes + + return ret; +}; + +Expr = Sizzle.selectors = { + + // Can be adjusted by the user + cacheLength: 50, + + createPseudo: markFunction, + + match: matchExpr, + + attrHandle: {}, + + find: {}, + + relative: { + ">": { dir: "parentNode", first: true }, + " ": { dir: "parentNode" }, + "+": { dir: "previousSibling", first: true }, + "~": { dir: "previousSibling" } + }, + + preFilter: { + "ATTR": function( match ) { + match[1] = match[1].replace( runescape, funescape ); + + // Move the given value to match[3] whether quoted or unquoted + match[3] = ( match[3] || match[4] || match[5] || "" ).replace( runescape, funescape ); + + if ( match[2] === "~=" ) { + match[3] = " " + match[3] + " "; + } + + return match.slice( 0, 4 ); + }, + + "CHILD": function( match ) { + /* matches from matchExpr["CHILD"] + 1 type (only|nth|...) + 2 what (child|of-type) + 3 argument (even|odd|\d*|\d*n([+-]\d+)?|...) + 4 xn-component of xn+y argument ([+-]?\d*n|) + 5 sign of xn-component + 6 x of xn-component + 7 sign of y-component + 8 y of y-component + */ + match[1] = match[1].toLowerCase(); + + if ( match[1].slice( 0, 3 ) === "nth" ) { + // nth-* requires argument + if ( !match[3] ) { + Sizzle.error( match[0] ); + } + + // numeric x and y parameters for Expr.filter.CHILD + // remember that false/true cast respectively to 0/1 + match[4] = +( match[4] ? match[5] + (match[6] || 1) : 2 * ( match[3] === "even" || match[3] === "odd" ) ); + match[5] = +( ( match[7] + match[8] ) || match[3] === "odd" ); + + // other types prohibit arguments + } else if ( match[3] ) { + Sizzle.error( match[0] ); + } + + return match; + }, + + "PSEUDO": function( match ) { + var excess, + unquoted = !match[6] && match[2]; + + if ( matchExpr["CHILD"].test( match[0] ) ) { + return null; + } + + // Accept quoted arguments as-is + if ( match[3] ) { + match[2] = match[4] || match[5] || ""; + + // Strip excess characters from unquoted arguments + } else if ( unquoted && rpseudo.test( unquoted ) && + // Get excess from tokenize (recursively) + (excess = tokenize( unquoted, true )) && + // advance to the next closing parenthesis + (excess = unquoted.indexOf( ")", unquoted.length - excess ) - unquoted.length) ) { + + // excess is a negative index + match[0] = match[0].slice( 0, excess ); + match[2] = unquoted.slice( 0, excess ); + } + + // Return only captures needed by the pseudo filter method (type and argument) + return match.slice( 0, 3 ); + } + }, + + filter: { + + "TAG": function( nodeNameSelector ) { + var nodeName = nodeNameSelector.replace( runescape, funescape ).toLowerCase(); + return nodeNameSelector === "*" ? + function() { return true; } : + function( elem ) { + return elem.nodeName && elem.nodeName.toLowerCase() === nodeName; + }; + }, + + "CLASS": function( className ) { + var pattern = classCache[ className + " " ]; + + return pattern || + (pattern = new RegExp( "(^|" + whitespace + ")" + className + "(" + whitespace + "|$)" )) && + classCache( className, function( elem ) { + return pattern.test( typeof elem.className === "string" && elem.className || typeof elem.getAttribute !== "undefined" && elem.getAttribute("class") || "" ); + }); + }, + + "ATTR": function( name, operator, check ) { + return function( elem ) { + var result = Sizzle.attr( elem, name ); + + if ( result == null ) { + return operator === "!="; + } + if ( !operator ) { + return true; + } + + result += ""; + + return operator === "=" ? result === check : + operator === "!=" ? result !== check : + operator === "^=" ? check && result.indexOf( check ) === 0 : + operator === "*=" ? check && result.indexOf( check ) > -1 : + operator === "$=" ? check && result.slice( -check.length ) === check : + operator === "~=" ? ( " " + result.replace( rwhitespace, " " ) + " " ).indexOf( check ) > -1 : + operator === "|=" ? result === check || result.slice( 0, check.length + 1 ) === check + "-" : + false; + }; + }, + + "CHILD": function( type, what, argument, first, last ) { + var simple = type.slice( 0, 3 ) !== "nth", + forward = type.slice( -4 ) !== "last", + ofType = what === "of-type"; + + return first === 1 && last === 0 ? + + // Shortcut for :nth-*(n) + function( elem ) { + return !!elem.parentNode; + } : + + function( elem, context, xml ) { + var cache, uniqueCache, outerCache, node, nodeIndex, start, + dir = simple !== forward ? "nextSibling" : "previousSibling", + parent = elem.parentNode, + name = ofType && elem.nodeName.toLowerCase(), + useCache = !xml && !ofType, + diff = false; + + if ( parent ) { + + // :(first|last|only)-(child|of-type) + if ( simple ) { + while ( dir ) { + node = elem; + while ( (node = node[ dir ]) ) { + if ( ofType ? + node.nodeName.toLowerCase() === name : + node.nodeType === 1 ) { + + return false; + } + } + // Reverse direction for :only-* (if we haven't yet done so) + start = dir = type === "only" && !start && "nextSibling"; + } + return true; + } + + start = [ forward ? parent.firstChild : parent.lastChild ]; + + // non-xml :nth-child(...) stores cache data on `parent` + if ( forward && useCache ) { + + // Seek `elem` from a previously-cached index + + // ...in a gzip-friendly way + node = parent; + outerCache = node[ expando ] || (node[ expando ] = {}); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ node.uniqueID ] || + (outerCache[ node.uniqueID ] = {}); + + cache = uniqueCache[ type ] || []; + nodeIndex = cache[ 0 ] === dirruns && cache[ 1 ]; + diff = nodeIndex && cache[ 2 ]; + node = nodeIndex && parent.childNodes[ nodeIndex ]; + + while ( (node = ++nodeIndex && node && node[ dir ] || + + // Fallback to seeking `elem` from the start + (diff = nodeIndex = 0) || start.pop()) ) { + + // When found, cache indexes on `parent` and break + if ( node.nodeType === 1 && ++diff && node === elem ) { + uniqueCache[ type ] = [ dirruns, nodeIndex, diff ]; + break; + } + } + + } else { + // Use previously-cached element index if available + if ( useCache ) { + // ...in a gzip-friendly way + node = elem; + outerCache = node[ expando ] || (node[ expando ] = {}); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ node.uniqueID ] || + (outerCache[ node.uniqueID ] = {}); + + cache = uniqueCache[ type ] || []; + nodeIndex = cache[ 0 ] === dirruns && cache[ 1 ]; + diff = nodeIndex; + } + + // xml :nth-child(...) + // or :nth-last-child(...) or :nth(-last)?-of-type(...) + if ( diff === false ) { + // Use the same loop as above to seek `elem` from the start + while ( (node = ++nodeIndex && node && node[ dir ] || + (diff = nodeIndex = 0) || start.pop()) ) { + + if ( ( ofType ? + node.nodeName.toLowerCase() === name : + node.nodeType === 1 ) && + ++diff ) { + + // Cache the index of each encountered element + if ( useCache ) { + outerCache = node[ expando ] || (node[ expando ] = {}); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ node.uniqueID ] || + (outerCache[ node.uniqueID ] = {}); + + uniqueCache[ type ] = [ dirruns, diff ]; + } + + if ( node === elem ) { + break; + } + } + } + } + } + + // Incorporate the offset, then check against cycle size + diff -= last; + return diff === first || ( diff % first === 0 && diff / first >= 0 ); + } + }; + }, + + "PSEUDO": function( pseudo, argument ) { + // pseudo-class names are case-insensitive + // http://www.w3.org/TR/selectors/#pseudo-classes + // Prioritize by case sensitivity in case custom pseudos are added with uppercase letters + // Remember that setFilters inherits from pseudos + var args, + fn = Expr.pseudos[ pseudo ] || Expr.setFilters[ pseudo.toLowerCase() ] || + Sizzle.error( "unsupported pseudo: " + pseudo ); + + // The user may use createPseudo to indicate that + // arguments are needed to create the filter function + // just as Sizzle does + if ( fn[ expando ] ) { + return fn( argument ); + } + + // But maintain support for old signatures + if ( fn.length > 1 ) { + args = [ pseudo, pseudo, "", argument ]; + return Expr.setFilters.hasOwnProperty( pseudo.toLowerCase() ) ? + markFunction(function( seed, matches ) { + var idx, + matched = fn( seed, argument ), + i = matched.length; + while ( i-- ) { + idx = indexOf( seed, matched[i] ); + seed[ idx ] = !( matches[ idx ] = matched[i] ); + } + }) : + function( elem ) { + return fn( elem, 0, args ); + }; + } + + return fn; + } + }, + + pseudos: { + // Potentially complex pseudos + "not": markFunction(function( selector ) { + // Trim the selector passed to compile + // to avoid treating leading and trailing + // spaces as combinators + var input = [], + results = [], + matcher = compile( selector.replace( rtrim, "$1" ) ); + + return matcher[ expando ] ? + markFunction(function( seed, matches, context, xml ) { + var elem, + unmatched = matcher( seed, null, xml, [] ), + i = seed.length; + + // Match elements unmatched by `matcher` + while ( i-- ) { + if ( (elem = unmatched[i]) ) { + seed[i] = !(matches[i] = elem); + } + } + }) : + function( elem, context, xml ) { + input[0] = elem; + matcher( input, null, xml, results ); + // Don't keep the element (issue #299) + input[0] = null; + return !results.pop(); + }; + }), + + "has": markFunction(function( selector ) { + return function( elem ) { + return Sizzle( selector, elem ).length > 0; + }; + }), + + "contains": markFunction(function( text ) { + text = text.replace( runescape, funescape ); + return function( elem ) { + return ( elem.textContent || getText( elem ) ).indexOf( text ) > -1; + }; + }), + + // "Whether an element is represented by a :lang() selector + // is based solely on the element's language value + // being equal to the identifier C, + // or beginning with the identifier C immediately followed by "-". + // The matching of C against the element's language value is performed case-insensitively. + // The identifier C does not have to be a valid language name." + // http://www.w3.org/TR/selectors/#lang-pseudo + "lang": markFunction( function( lang ) { + // lang value must be a valid identifier + if ( !ridentifier.test(lang || "") ) { + Sizzle.error( "unsupported lang: " + lang ); + } + lang = lang.replace( runescape, funescape ).toLowerCase(); + return function( elem ) { + var elemLang; + do { + if ( (elemLang = documentIsHTML ? + elem.lang : + elem.getAttribute("xml:lang") || elem.getAttribute("lang")) ) { + + elemLang = elemLang.toLowerCase(); + return elemLang === lang || elemLang.indexOf( lang + "-" ) === 0; + } + } while ( (elem = elem.parentNode) && elem.nodeType === 1 ); + return false; + }; + }), + + // Miscellaneous + "target": function( elem ) { + var hash = window.location && window.location.hash; + return hash && hash.slice( 1 ) === elem.id; + }, + + "root": function( elem ) { + return elem === docElem; + }, + + "focus": function( elem ) { + return elem === document.activeElement && (!document.hasFocus || document.hasFocus()) && !!(elem.type || elem.href || ~elem.tabIndex); + }, + + // Boolean properties + "enabled": createDisabledPseudo( false ), + "disabled": createDisabledPseudo( true ), + + "checked": function( elem ) { + // In CSS3, :checked should return both checked and selected elements + // http://www.w3.org/TR/2011/REC-css3-selectors-20110929/#checked + var nodeName = elem.nodeName.toLowerCase(); + return (nodeName === "input" && !!elem.checked) || (nodeName === "option" && !!elem.selected); + }, + + "selected": function( elem ) { + // Accessing this property makes selected-by-default + // options in Safari work properly + if ( elem.parentNode ) { + elem.parentNode.selectedIndex; + } + + return elem.selected === true; + }, + + // Contents + "empty": function( elem ) { + // http://www.w3.org/TR/selectors/#empty-pseudo + // :empty is negated by element (1) or content nodes (text: 3; cdata: 4; entity ref: 5), + // but not by others (comment: 8; processing instruction: 7; etc.) + // nodeType < 6 works because attributes (2) do not appear as children + for ( elem = elem.firstChild; elem; elem = elem.nextSibling ) { + if ( elem.nodeType < 6 ) { + return false; + } + } + return true; + }, + + "parent": function( elem ) { + return !Expr.pseudos["empty"]( elem ); + }, + + // Element/input types + "header": function( elem ) { + return rheader.test( elem.nodeName ); + }, + + "input": function( elem ) { + return rinputs.test( elem.nodeName ); + }, + + "button": function( elem ) { + var name = elem.nodeName.toLowerCase(); + return name === "input" && elem.type === "button" || name === "button"; + }, + + "text": function( elem ) { + var attr; + return elem.nodeName.toLowerCase() === "input" && + elem.type === "text" && + + // Support: IE<8 + // New HTML5 attribute values (e.g., "search") appear with elem.type === "text" + ( (attr = elem.getAttribute("type")) == null || attr.toLowerCase() === "text" ); + }, + + // Position-in-collection + "first": createPositionalPseudo(function() { + return [ 0 ]; + }), + + "last": createPositionalPseudo(function( matchIndexes, length ) { + return [ length - 1 ]; + }), + + "eq": createPositionalPseudo(function( matchIndexes, length, argument ) { + return [ argument < 0 ? argument + length : argument ]; + }), + + "even": createPositionalPseudo(function( matchIndexes, length ) { + var i = 0; + for ( ; i < length; i += 2 ) { + matchIndexes.push( i ); + } + return matchIndexes; + }), + + "odd": createPositionalPseudo(function( matchIndexes, length ) { + var i = 1; + for ( ; i < length; i += 2 ) { + matchIndexes.push( i ); + } + return matchIndexes; + }), + + "lt": createPositionalPseudo(function( matchIndexes, length, argument ) { + var i = argument < 0 ? + argument + length : + argument > length ? + length : + argument; + for ( ; --i >= 0; ) { + matchIndexes.push( i ); + } + return matchIndexes; + }), + + "gt": createPositionalPseudo(function( matchIndexes, length, argument ) { + var i = argument < 0 ? argument + length : argument; + for ( ; ++i < length; ) { + matchIndexes.push( i ); + } + return matchIndexes; + }) + } +}; + +Expr.pseudos["nth"] = Expr.pseudos["eq"]; + +// Add button/input type pseudos +for ( i in { radio: true, checkbox: true, file: true, password: true, image: true } ) { + Expr.pseudos[ i ] = createInputPseudo( i ); +} +for ( i in { submit: true, reset: true } ) { + Expr.pseudos[ i ] = createButtonPseudo( i ); +} + +// Easy API for creating new setFilters +function setFilters() {} +setFilters.prototype = Expr.filters = Expr.pseudos; +Expr.setFilters = new setFilters(); + +tokenize = Sizzle.tokenize = function( selector, parseOnly ) { + var matched, match, tokens, type, + soFar, groups, preFilters, + cached = tokenCache[ selector + " " ]; + + if ( cached ) { + return parseOnly ? 0 : cached.slice( 0 ); + } + + soFar = selector; + groups = []; + preFilters = Expr.preFilter; + + while ( soFar ) { + + // Comma and first run + if ( !matched || (match = rcomma.exec( soFar )) ) { + if ( match ) { + // Don't consume trailing commas as valid + soFar = soFar.slice( match[0].length ) || soFar; + } + groups.push( (tokens = []) ); + } + + matched = false; + + // Combinators + if ( (match = rcombinators.exec( soFar )) ) { + matched = match.shift(); + tokens.push({ + value: matched, + // Cast descendant combinators to space + type: match[0].replace( rtrim, " " ) + }); + soFar = soFar.slice( matched.length ); + } + + // Filters + for ( type in Expr.filter ) { + if ( (match = matchExpr[ type ].exec( soFar )) && (!preFilters[ type ] || + (match = preFilters[ type ]( match ))) ) { + matched = match.shift(); + tokens.push({ + value: matched, + type: type, + matches: match + }); + soFar = soFar.slice( matched.length ); + } + } + + if ( !matched ) { + break; + } + } + + // Return the length of the invalid excess + // if we're just parsing + // Otherwise, throw an error or return tokens + return parseOnly ? + soFar.length : + soFar ? + Sizzle.error( selector ) : + // Cache the tokens + tokenCache( selector, groups ).slice( 0 ); +}; + +function toSelector( tokens ) { + var i = 0, + len = tokens.length, + selector = ""; + for ( ; i < len; i++ ) { + selector += tokens[i].value; + } + return selector; +} + +function addCombinator( matcher, combinator, base ) { + var dir = combinator.dir, + skip = combinator.next, + key = skip || dir, + checkNonElements = base && key === "parentNode", + doneName = done++; + + return combinator.first ? + // Check against closest ancestor/preceding element + function( elem, context, xml ) { + while ( (elem = elem[ dir ]) ) { + if ( elem.nodeType === 1 || checkNonElements ) { + return matcher( elem, context, xml ); + } + } + return false; + } : + + // Check against all ancestor/preceding elements + function( elem, context, xml ) { + var oldCache, uniqueCache, outerCache, + newCache = [ dirruns, doneName ]; + + // We can't set arbitrary data on XML nodes, so they don't benefit from combinator caching + if ( xml ) { + while ( (elem = elem[ dir ]) ) { + if ( elem.nodeType === 1 || checkNonElements ) { + if ( matcher( elem, context, xml ) ) { + return true; + } + } + } + } else { + while ( (elem = elem[ dir ]) ) { + if ( elem.nodeType === 1 || checkNonElements ) { + outerCache = elem[ expando ] || (elem[ expando ] = {}); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ elem.uniqueID ] || (outerCache[ elem.uniqueID ] = {}); + + if ( skip && skip === elem.nodeName.toLowerCase() ) { + elem = elem[ dir ] || elem; + } else if ( (oldCache = uniqueCache[ key ]) && + oldCache[ 0 ] === dirruns && oldCache[ 1 ] === doneName ) { + + // Assign to newCache so results back-propagate to previous elements + return (newCache[ 2 ] = oldCache[ 2 ]); + } else { + // Reuse newcache so results back-propagate to previous elements + uniqueCache[ key ] = newCache; + + // A match means we're done; a fail means we have to keep checking + if ( (newCache[ 2 ] = matcher( elem, context, xml )) ) { + return true; + } + } + } + } + } + return false; + }; +} + +function elementMatcher( matchers ) { + return matchers.length > 1 ? + function( elem, context, xml ) { + var i = matchers.length; + while ( i-- ) { + if ( !matchers[i]( elem, context, xml ) ) { + return false; + } + } + return true; + } : + matchers[0]; +} + +function multipleContexts( selector, contexts, results ) { + var i = 0, + len = contexts.length; + for ( ; i < len; i++ ) { + Sizzle( selector, contexts[i], results ); + } + return results; +} + +function condense( unmatched, map, filter, context, xml ) { + var elem, + newUnmatched = [], + i = 0, + len = unmatched.length, + mapped = map != null; + + for ( ; i < len; i++ ) { + if ( (elem = unmatched[i]) ) { + if ( !filter || filter( elem, context, xml ) ) { + newUnmatched.push( elem ); + if ( mapped ) { + map.push( i ); + } + } + } + } + + return newUnmatched; +} + +function setMatcher( preFilter, selector, matcher, postFilter, postFinder, postSelector ) { + if ( postFilter && !postFilter[ expando ] ) { + postFilter = setMatcher( postFilter ); + } + if ( postFinder && !postFinder[ expando ] ) { + postFinder = setMatcher( postFinder, postSelector ); + } + return markFunction(function( seed, results, context, xml ) { + var temp, i, elem, + preMap = [], + postMap = [], + preexisting = results.length, + + // Get initial elements from seed or context + elems = seed || multipleContexts( selector || "*", context.nodeType ? [ context ] : context, [] ), + + // Prefilter to get matcher input, preserving a map for seed-results synchronization + matcherIn = preFilter && ( seed || !selector ) ? + condense( elems, preMap, preFilter, context, xml ) : + elems, + + matcherOut = matcher ? + // If we have a postFinder, or filtered seed, or non-seed postFilter or preexisting results, + postFinder || ( seed ? preFilter : preexisting || postFilter ) ? + + // ...intermediate processing is necessary + [] : + + // ...otherwise use results directly + results : + matcherIn; + + // Find primary matches + if ( matcher ) { + matcher( matcherIn, matcherOut, context, xml ); + } + + // Apply postFilter + if ( postFilter ) { + temp = condense( matcherOut, postMap ); + postFilter( temp, [], context, xml ); + + // Un-match failing elements by moving them back to matcherIn + i = temp.length; + while ( i-- ) { + if ( (elem = temp[i]) ) { + matcherOut[ postMap[i] ] = !(matcherIn[ postMap[i] ] = elem); + } + } + } + + if ( seed ) { + if ( postFinder || preFilter ) { + if ( postFinder ) { + // Get the final matcherOut by condensing this intermediate into postFinder contexts + temp = []; + i = matcherOut.length; + while ( i-- ) { + if ( (elem = matcherOut[i]) ) { + // Restore matcherIn since elem is not yet a final match + temp.push( (matcherIn[i] = elem) ); + } + } + postFinder( null, (matcherOut = []), temp, xml ); + } + + // Move matched elements from seed to results to keep them synchronized + i = matcherOut.length; + while ( i-- ) { + if ( (elem = matcherOut[i]) && + (temp = postFinder ? indexOf( seed, elem ) : preMap[i]) > -1 ) { + + seed[temp] = !(results[temp] = elem); + } + } + } + + // Add elements to results, through postFinder if defined + } else { + matcherOut = condense( + matcherOut === results ? + matcherOut.splice( preexisting, matcherOut.length ) : + matcherOut + ); + if ( postFinder ) { + postFinder( null, results, matcherOut, xml ); + } else { + push.apply( results, matcherOut ); + } + } + }); +} + +function matcherFromTokens( tokens ) { + var checkContext, matcher, j, + len = tokens.length, + leadingRelative = Expr.relative[ tokens[0].type ], + implicitRelative = leadingRelative || Expr.relative[" "], + i = leadingRelative ? 1 : 0, + + // The foundational matcher ensures that elements are reachable from top-level context(s) + matchContext = addCombinator( function( elem ) { + return elem === checkContext; + }, implicitRelative, true ), + matchAnyContext = addCombinator( function( elem ) { + return indexOf( checkContext, elem ) > -1; + }, implicitRelative, true ), + matchers = [ function( elem, context, xml ) { + var ret = ( !leadingRelative && ( xml || context !== outermostContext ) ) || ( + (checkContext = context).nodeType ? + matchContext( elem, context, xml ) : + matchAnyContext( elem, context, xml ) ); + // Avoid hanging onto element (issue #299) + checkContext = null; + return ret; + } ]; + + for ( ; i < len; i++ ) { + if ( (matcher = Expr.relative[ tokens[i].type ]) ) { + matchers = [ addCombinator(elementMatcher( matchers ), matcher) ]; + } else { + matcher = Expr.filter[ tokens[i].type ].apply( null, tokens[i].matches ); + + // Return special upon seeing a positional matcher + if ( matcher[ expando ] ) { + // Find the next relative operator (if any) for proper handling + j = ++i; + for ( ; j < len; j++ ) { + if ( Expr.relative[ tokens[j].type ] ) { + break; + } + } + return setMatcher( + i > 1 && elementMatcher( matchers ), + i > 1 && toSelector( + // If the preceding token was a descendant combinator, insert an implicit any-element `*` + tokens.slice( 0, i - 1 ).concat({ value: tokens[ i - 2 ].type === " " ? "*" : "" }) + ).replace( rtrim, "$1" ), + matcher, + i < j && matcherFromTokens( tokens.slice( i, j ) ), + j < len && matcherFromTokens( (tokens = tokens.slice( j )) ), + j < len && toSelector( tokens ) + ); + } + matchers.push( matcher ); + } + } + + return elementMatcher( matchers ); +} + +function matcherFromGroupMatchers( elementMatchers, setMatchers ) { + var bySet = setMatchers.length > 0, + byElement = elementMatchers.length > 0, + superMatcher = function( seed, context, xml, results, outermost ) { + var elem, j, matcher, + matchedCount = 0, + i = "0", + unmatched = seed && [], + setMatched = [], + contextBackup = outermostContext, + // We must always have either seed elements or outermost context + elems = seed || byElement && Expr.find["TAG"]( "*", outermost ), + // Use integer dirruns iff this is the outermost matcher + dirrunsUnique = (dirruns += contextBackup == null ? 1 : Math.random() || 0.1), + len = elems.length; + + if ( outermost ) { + outermostContext = context === document || context || outermost; + } + + // Add elements passing elementMatchers directly to results + // Support: IE<9, Safari + // Tolerate NodeList properties (IE: "length"; Safari: ) matching elements by id + for ( ; i !== len && (elem = elems[i]) != null; i++ ) { + if ( byElement && elem ) { + j = 0; + if ( !context && elem.ownerDocument !== document ) { + setDocument( elem ); + xml = !documentIsHTML; + } + while ( (matcher = elementMatchers[j++]) ) { + if ( matcher( elem, context || document, xml) ) { + results.push( elem ); + break; + } + } + if ( outermost ) { + dirruns = dirrunsUnique; + } + } + + // Track unmatched elements for set filters + if ( bySet ) { + // They will have gone through all possible matchers + if ( (elem = !matcher && elem) ) { + matchedCount--; + } + + // Lengthen the array for every element, matched or not + if ( seed ) { + unmatched.push( elem ); + } + } + } + + // `i` is now the count of elements visited above, and adding it to `matchedCount` + // makes the latter nonnegative. + matchedCount += i; + + // Apply set filters to unmatched elements + // NOTE: This can be skipped if there are no unmatched elements (i.e., `matchedCount` + // equals `i`), unless we didn't visit _any_ elements in the above loop because we have + // no element matchers and no seed. + // Incrementing an initially-string "0" `i` allows `i` to remain a string only in that + // case, which will result in a "00" `matchedCount` that differs from `i` but is also + // numerically zero. + if ( bySet && i !== matchedCount ) { + j = 0; + while ( (matcher = setMatchers[j++]) ) { + matcher( unmatched, setMatched, context, xml ); + } + + if ( seed ) { + // Reintegrate element matches to eliminate the need for sorting + if ( matchedCount > 0 ) { + while ( i-- ) { + if ( !(unmatched[i] || setMatched[i]) ) { + setMatched[i] = pop.call( results ); + } + } + } + + // Discard index placeholder values to get only actual matches + setMatched = condense( setMatched ); + } + + // Add matches to results + push.apply( results, setMatched ); + + // Seedless set matches succeeding multiple successful matchers stipulate sorting + if ( outermost && !seed && setMatched.length > 0 && + ( matchedCount + setMatchers.length ) > 1 ) { + + Sizzle.uniqueSort( results ); + } + } + + // Override manipulation of globals by nested matchers + if ( outermost ) { + dirruns = dirrunsUnique; + outermostContext = contextBackup; + } + + return unmatched; + }; + + return bySet ? + markFunction( superMatcher ) : + superMatcher; +} + +compile = Sizzle.compile = function( selector, match /* Internal Use Only */ ) { + var i, + setMatchers = [], + elementMatchers = [], + cached = compilerCache[ selector + " " ]; + + if ( !cached ) { + // Generate a function of recursive functions that can be used to check each element + if ( !match ) { + match = tokenize( selector ); + } + i = match.length; + while ( i-- ) { + cached = matcherFromTokens( match[i] ); + if ( cached[ expando ] ) { + setMatchers.push( cached ); + } else { + elementMatchers.push( cached ); + } + } + + // Cache the compiled function + cached = compilerCache( selector, matcherFromGroupMatchers( elementMatchers, setMatchers ) ); + + // Save selector and tokenization + cached.selector = selector; + } + return cached; +}; + +/** + * A low-level selection function that works with Sizzle's compiled + * selector functions + * @param {String|Function} selector A selector or a pre-compiled + * selector function built with Sizzle.compile + * @param {Element} context + * @param {Array} [results] + * @param {Array} [seed] A set of elements to match against + */ +select = Sizzle.select = function( selector, context, results, seed ) { + var i, tokens, token, type, find, + compiled = typeof selector === "function" && selector, + match = !seed && tokenize( (selector = compiled.selector || selector) ); + + results = results || []; + + // Try to minimize operations if there is only one selector in the list and no seed + // (the latter of which guarantees us context) + if ( match.length === 1 ) { + + // Reduce context if the leading compound selector is an ID + tokens = match[0] = match[0].slice( 0 ); + if ( tokens.length > 2 && (token = tokens[0]).type === "ID" && + context.nodeType === 9 && documentIsHTML && Expr.relative[ tokens[1].type ] ) { + + context = ( Expr.find["ID"]( token.matches[0].replace(runescape, funescape), context ) || [] )[0]; + if ( !context ) { + return results; + + // Precompiled matchers will still verify ancestry, so step up a level + } else if ( compiled ) { + context = context.parentNode; + } + + selector = selector.slice( tokens.shift().value.length ); + } + + // Fetch a seed set for right-to-left matching + i = matchExpr["needsContext"].test( selector ) ? 0 : tokens.length; + while ( i-- ) { + token = tokens[i]; + + // Abort if we hit a combinator + if ( Expr.relative[ (type = token.type) ] ) { + break; + } + if ( (find = Expr.find[ type ]) ) { + // Search, expanding context for leading sibling combinators + if ( (seed = find( + token.matches[0].replace( runescape, funescape ), + rsibling.test( tokens[0].type ) && testContext( context.parentNode ) || context + )) ) { + + // If seed is empty or no tokens remain, we can return early + tokens.splice( i, 1 ); + selector = seed.length && toSelector( tokens ); + if ( !selector ) { + push.apply( results, seed ); + return results; + } + + break; + } + } + } + } + + // Compile and execute a filtering function if one is not provided + // Provide `match` to avoid retokenization if we modified the selector above + ( compiled || compile( selector, match ) )( + seed, + context, + !documentIsHTML, + results, + !context || rsibling.test( selector ) && testContext( context.parentNode ) || context + ); + return results; +}; + +// One-time assignments + +// Sort stability +support.sortStable = expando.split("").sort( sortOrder ).join("") === expando; + +// Support: Chrome 14-35+ +// Always assume duplicates if they aren't passed to the comparison function +support.detectDuplicates = !!hasDuplicate; + +// Initialize against the default document +setDocument(); + +// Support: Webkit<537.32 - Safari 6.0.3/Chrome 25 (fixed in Chrome 27) +// Detached nodes confoundingly follow *each other* +support.sortDetached = assert(function( el ) { + // Should return 1, but returns 4 (following) + return el.compareDocumentPosition( document.createElement("fieldset") ) & 1; +}); + +// Support: IE<8 +// Prevent attribute/property "interpolation" +// https://msdn.microsoft.com/en-us/library/ms536429%28VS.85%29.aspx +if ( !assert(function( el ) { + el.innerHTML = ""; + return el.firstChild.getAttribute("href") === "#" ; +}) ) { + addHandle( "type|href|height|width", function( elem, name, isXML ) { + if ( !isXML ) { + return elem.getAttribute( name, name.toLowerCase() === "type" ? 1 : 2 ); + } + }); +} + +// Support: IE<9 +// Use defaultValue in place of getAttribute("value") +if ( !support.attributes || !assert(function( el ) { + el.innerHTML = ""; + el.firstChild.setAttribute( "value", "" ); + return el.firstChild.getAttribute( "value" ) === ""; +}) ) { + addHandle( "value", function( elem, name, isXML ) { + if ( !isXML && elem.nodeName.toLowerCase() === "input" ) { + return elem.defaultValue; + } + }); +} + +// Support: IE<9 +// Use getAttributeNode to fetch booleans when getAttribute lies +if ( !assert(function( el ) { + return el.getAttribute("disabled") == null; +}) ) { + addHandle( booleans, function( elem, name, isXML ) { + var val; + if ( !isXML ) { + return elem[ name ] === true ? name.toLowerCase() : + (val = elem.getAttributeNode( name )) && val.specified ? + val.value : + null; + } + }); +} + +return Sizzle; + +})( window ); + + + +jQuery.find = Sizzle; +jQuery.expr = Sizzle.selectors; + +// Deprecated +jQuery.expr[ ":" ] = jQuery.expr.pseudos; +jQuery.uniqueSort = jQuery.unique = Sizzle.uniqueSort; +jQuery.text = Sizzle.getText; +jQuery.isXMLDoc = Sizzle.isXML; +jQuery.contains = Sizzle.contains; +jQuery.escapeSelector = Sizzle.escape; + + + + +var dir = function( elem, dir, until ) { + var matched = [], + truncate = until !== undefined; + + while ( ( elem = elem[ dir ] ) && elem.nodeType !== 9 ) { + if ( elem.nodeType === 1 ) { + if ( truncate && jQuery( elem ).is( until ) ) { + break; + } + matched.push( elem ); + } + } + return matched; +}; + + +var siblings = function( n, elem ) { + var matched = []; + + for ( ; n; n = n.nextSibling ) { + if ( n.nodeType === 1 && n !== elem ) { + matched.push( n ); + } + } + + return matched; +}; + + +var rneedsContext = jQuery.expr.match.needsContext; + + + +function nodeName( elem, name ) { + + return elem.nodeName && elem.nodeName.toLowerCase() === name.toLowerCase(); + +}; +var rsingleTag = ( /^<([a-z][^\/\0>:\x20\t\r\n\f]*)[\x20\t\r\n\f]*\/?>(?:<\/\1>|)$/i ); + + + +// Implement the identical functionality for filter and not +function winnow( elements, qualifier, not ) { + if ( isFunction( qualifier ) ) { + return jQuery.grep( elements, function( elem, i ) { + return !!qualifier.call( elem, i, elem ) !== not; + } ); + } + + // Single element + if ( qualifier.nodeType ) { + return jQuery.grep( elements, function( elem ) { + return ( elem === qualifier ) !== not; + } ); + } + + // Arraylike of elements (jQuery, arguments, Array) + if ( typeof qualifier !== "string" ) { + return jQuery.grep( elements, function( elem ) { + return ( indexOf.call( qualifier, elem ) > -1 ) !== not; + } ); + } + + // Filtered directly for both simple and complex selectors + return jQuery.filter( qualifier, elements, not ); +} + +jQuery.filter = function( expr, elems, not ) { + var elem = elems[ 0 ]; + + if ( not ) { + expr = ":not(" + expr + ")"; + } + + if ( elems.length === 1 && elem.nodeType === 1 ) { + return jQuery.find.matchesSelector( elem, expr ) ? [ elem ] : []; + } + + return jQuery.find.matches( expr, jQuery.grep( elems, function( elem ) { + return elem.nodeType === 1; + } ) ); +}; + +jQuery.fn.extend( { + find: function( selector ) { + var i, ret, + len = this.length, + self = this; + + if ( typeof selector !== "string" ) { + return this.pushStack( jQuery( selector ).filter( function() { + for ( i = 0; i < len; i++ ) { + if ( jQuery.contains( self[ i ], this ) ) { + return true; + } + } + } ) ); + } + + ret = this.pushStack( [] ); + + for ( i = 0; i < len; i++ ) { + jQuery.find( selector, self[ i ], ret ); + } + + return len > 1 ? jQuery.uniqueSort( ret ) : ret; + }, + filter: function( selector ) { + return this.pushStack( winnow( this, selector || [], false ) ); + }, + not: function( selector ) { + return this.pushStack( winnow( this, selector || [], true ) ); + }, + is: function( selector ) { + return !!winnow( + this, + + // If this is a positional/relative selector, check membership in the returned set + // so $("p:first").is("p:last") won't return true for a doc with two "p". + typeof selector === "string" && rneedsContext.test( selector ) ? + jQuery( selector ) : + selector || [], + false + ).length; + } +} ); + + +// Initialize a jQuery object + + +// A central reference to the root jQuery(document) +var rootjQuery, + + // A simple way to check for HTML strings + // Prioritize #id over to avoid XSS via location.hash (#9521) + // Strict HTML recognition (#11290: must start with <) + // Shortcut simple #id case for speed + rquickExpr = /^(?:\s*(<[\w\W]+>)[^>]*|#([\w-]+))$/, + + init = jQuery.fn.init = function( selector, context, root ) { + var match, elem; + + // HANDLE: $(""), $(null), $(undefined), $(false) + if ( !selector ) { + return this; + } + + // Method init() accepts an alternate rootjQuery + // so migrate can support jQuery.sub (gh-2101) + root = root || rootjQuery; + + // Handle HTML strings + if ( typeof selector === "string" ) { + if ( selector[ 0 ] === "<" && + selector[ selector.length - 1 ] === ">" && + selector.length >= 3 ) { + + // Assume that strings that start and end with <> are HTML and skip the regex check + match = [ null, selector, null ]; + + } else { + match = rquickExpr.exec( selector ); + } + + // Match html or make sure no context is specified for #id + if ( match && ( match[ 1 ] || !context ) ) { + + // HANDLE: $(html) -> $(array) + if ( match[ 1 ] ) { + context = context instanceof jQuery ? context[ 0 ] : context; + + // Option to run scripts is true for back-compat + // Intentionally let the error be thrown if parseHTML is not present + jQuery.merge( this, jQuery.parseHTML( + match[ 1 ], + context && context.nodeType ? context.ownerDocument || context : document, + true + ) ); + + // HANDLE: $(html, props) + if ( rsingleTag.test( match[ 1 ] ) && jQuery.isPlainObject( context ) ) { + for ( match in context ) { + + // Properties of context are called as methods if possible + if ( isFunction( this[ match ] ) ) { + this[ match ]( context[ match ] ); + + // ...and otherwise set as attributes + } else { + this.attr( match, context[ match ] ); + } + } + } + + return this; + + // HANDLE: $(#id) + } else { + elem = document.getElementById( match[ 2 ] ); + + if ( elem ) { + + // Inject the element directly into the jQuery object + this[ 0 ] = elem; + this.length = 1; + } + return this; + } + + // HANDLE: $(expr, $(...)) + } else if ( !context || context.jquery ) { + return ( context || root ).find( selector ); + + // HANDLE: $(expr, context) + // (which is just equivalent to: $(context).find(expr) + } else { + return this.constructor( context ).find( selector ); + } + + // HANDLE: $(DOMElement) + } else if ( selector.nodeType ) { + this[ 0 ] = selector; + this.length = 1; + return this; + + // HANDLE: $(function) + // Shortcut for document ready + } else if ( isFunction( selector ) ) { + return root.ready !== undefined ? + root.ready( selector ) : + + // Execute immediately if ready is not present + selector( jQuery ); + } + + return jQuery.makeArray( selector, this ); + }; + +// Give the init function the jQuery prototype for later instantiation +init.prototype = jQuery.fn; + +// Initialize central reference +rootjQuery = jQuery( document ); + + +var rparentsprev = /^(?:parents|prev(?:Until|All))/, + + // Methods guaranteed to produce a unique set when starting from a unique set + guaranteedUnique = { + children: true, + contents: true, + next: true, + prev: true + }; + +jQuery.fn.extend( { + has: function( target ) { + var targets = jQuery( target, this ), + l = targets.length; + + return this.filter( function() { + var i = 0; + for ( ; i < l; i++ ) { + if ( jQuery.contains( this, targets[ i ] ) ) { + return true; + } + } + } ); + }, + + closest: function( selectors, context ) { + var cur, + i = 0, + l = this.length, + matched = [], + targets = typeof selectors !== "string" && jQuery( selectors ); + + // Positional selectors never match, since there's no _selection_ context + if ( !rneedsContext.test( selectors ) ) { + for ( ; i < l; i++ ) { + for ( cur = this[ i ]; cur && cur !== context; cur = cur.parentNode ) { + + // Always skip document fragments + if ( cur.nodeType < 11 && ( targets ? + targets.index( cur ) > -1 : + + // Don't pass non-elements to Sizzle + cur.nodeType === 1 && + jQuery.find.matchesSelector( cur, selectors ) ) ) { + + matched.push( cur ); + break; + } + } + } + } + + return this.pushStack( matched.length > 1 ? jQuery.uniqueSort( matched ) : matched ); + }, + + // Determine the position of an element within the set + index: function( elem ) { + + // No argument, return index in parent + if ( !elem ) { + return ( this[ 0 ] && this[ 0 ].parentNode ) ? this.first().prevAll().length : -1; + } + + // Index in selector + if ( typeof elem === "string" ) { + return indexOf.call( jQuery( elem ), this[ 0 ] ); + } + + // Locate the position of the desired element + return indexOf.call( this, + + // If it receives a jQuery object, the first element is used + elem.jquery ? elem[ 0 ] : elem + ); + }, + + add: function( selector, context ) { + return this.pushStack( + jQuery.uniqueSort( + jQuery.merge( this.get(), jQuery( selector, context ) ) + ) + ); + }, + + addBack: function( selector ) { + return this.add( selector == null ? + this.prevObject : this.prevObject.filter( selector ) + ); + } +} ); + +function sibling( cur, dir ) { + while ( ( cur = cur[ dir ] ) && cur.nodeType !== 1 ) {} + return cur; +} + +jQuery.each( { + parent: function( elem ) { + var parent = elem.parentNode; + return parent && parent.nodeType !== 11 ? parent : null; + }, + parents: function( elem ) { + return dir( elem, "parentNode" ); + }, + parentsUntil: function( elem, i, until ) { + return dir( elem, "parentNode", until ); + }, + next: function( elem ) { + return sibling( elem, "nextSibling" ); + }, + prev: function( elem ) { + return sibling( elem, "previousSibling" ); + }, + nextAll: function( elem ) { + return dir( elem, "nextSibling" ); + }, + prevAll: function( elem ) { + return dir( elem, "previousSibling" ); + }, + nextUntil: function( elem, i, until ) { + return dir( elem, "nextSibling", until ); + }, + prevUntil: function( elem, i, until ) { + return dir( elem, "previousSibling", until ); + }, + siblings: function( elem ) { + return siblings( ( elem.parentNode || {} ).firstChild, elem ); + }, + children: function( elem ) { + return siblings( elem.firstChild ); + }, + contents: function( elem ) { + if ( typeof elem.contentDocument !== "undefined" ) { + return elem.contentDocument; + } + + // Support: IE 9 - 11 only, iOS 7 only, Android Browser <=4.3 only + // Treat the template element as a regular one in browsers that + // don't support it. + if ( nodeName( elem, "template" ) ) { + elem = elem.content || elem; + } + + return jQuery.merge( [], elem.childNodes ); + } +}, function( name, fn ) { + jQuery.fn[ name ] = function( until, selector ) { + var matched = jQuery.map( this, fn, until ); + + if ( name.slice( -5 ) !== "Until" ) { + selector = until; + } + + if ( selector && typeof selector === "string" ) { + matched = jQuery.filter( selector, matched ); + } + + if ( this.length > 1 ) { + + // Remove duplicates + if ( !guaranteedUnique[ name ] ) { + jQuery.uniqueSort( matched ); + } + + // Reverse order for parents* and prev-derivatives + if ( rparentsprev.test( name ) ) { + matched.reverse(); + } + } + + return this.pushStack( matched ); + }; +} ); +var rnothtmlwhite = ( /[^\x20\t\r\n\f]+/g ); + + + +// Convert String-formatted options into Object-formatted ones +function createOptions( options ) { + var object = {}; + jQuery.each( options.match( rnothtmlwhite ) || [], function( _, flag ) { + object[ flag ] = true; + } ); + return object; +} + +/* + * Create a callback list using the following parameters: + * + * options: an optional list of space-separated options that will change how + * the callback list behaves or a more traditional option object + * + * By default a callback list will act like an event callback list and can be + * "fired" multiple times. + * + * Possible options: + * + * once: will ensure the callback list can only be fired once (like a Deferred) + * + * memory: will keep track of previous values and will call any callback added + * after the list has been fired right away with the latest "memorized" + * values (like a Deferred) + * + * unique: will ensure a callback can only be added once (no duplicate in the list) + * + * stopOnFalse: interrupt callings when a callback returns false + * + */ +jQuery.Callbacks = function( options ) { + + // Convert options from String-formatted to Object-formatted if needed + // (we check in cache first) + options = typeof options === "string" ? + createOptions( options ) : + jQuery.extend( {}, options ); + + var // Flag to know if list is currently firing + firing, + + // Last fire value for non-forgettable lists + memory, + + // Flag to know if list was already fired + fired, + + // Flag to prevent firing + locked, + + // Actual callback list + list = [], + + // Queue of execution data for repeatable lists + queue = [], + + // Index of currently firing callback (modified by add/remove as needed) + firingIndex = -1, + + // Fire callbacks + fire = function() { + + // Enforce single-firing + locked = locked || options.once; + + // Execute callbacks for all pending executions, + // respecting firingIndex overrides and runtime changes + fired = firing = true; + for ( ; queue.length; firingIndex = -1 ) { + memory = queue.shift(); + while ( ++firingIndex < list.length ) { + + // Run callback and check for early termination + if ( list[ firingIndex ].apply( memory[ 0 ], memory[ 1 ] ) === false && + options.stopOnFalse ) { + + // Jump to end and forget the data so .add doesn't re-fire + firingIndex = list.length; + memory = false; + } + } + } + + // Forget the data if we're done with it + if ( !options.memory ) { + memory = false; + } + + firing = false; + + // Clean up if we're done firing for good + if ( locked ) { + + // Keep an empty list if we have data for future add calls + if ( memory ) { + list = []; + + // Otherwise, this object is spent + } else { + list = ""; + } + } + }, + + // Actual Callbacks object + self = { + + // Add a callback or a collection of callbacks to the list + add: function() { + if ( list ) { + + // If we have memory from a past run, we should fire after adding + if ( memory && !firing ) { + firingIndex = list.length - 1; + queue.push( memory ); + } + + ( function add( args ) { + jQuery.each( args, function( _, arg ) { + if ( isFunction( arg ) ) { + if ( !options.unique || !self.has( arg ) ) { + list.push( arg ); + } + } else if ( arg && arg.length && toType( arg ) !== "string" ) { + + // Inspect recursively + add( arg ); + } + } ); + } )( arguments ); + + if ( memory && !firing ) { + fire(); + } + } + return this; + }, + + // Remove a callback from the list + remove: function() { + jQuery.each( arguments, function( _, arg ) { + var index; + while ( ( index = jQuery.inArray( arg, list, index ) ) > -1 ) { + list.splice( index, 1 ); + + // Handle firing indexes + if ( index <= firingIndex ) { + firingIndex--; + } + } + } ); + return this; + }, + + // Check if a given callback is in the list. + // If no argument is given, return whether or not list has callbacks attached. + has: function( fn ) { + return fn ? + jQuery.inArray( fn, list ) > -1 : + list.length > 0; + }, + + // Remove all callbacks from the list + empty: function() { + if ( list ) { + list = []; + } + return this; + }, + + // Disable .fire and .add + // Abort any current/pending executions + // Clear all callbacks and values + disable: function() { + locked = queue = []; + list = memory = ""; + return this; + }, + disabled: function() { + return !list; + }, + + // Disable .fire + // Also disable .add unless we have memory (since it would have no effect) + // Abort any pending executions + lock: function() { + locked = queue = []; + if ( !memory && !firing ) { + list = memory = ""; + } + return this; + }, + locked: function() { + return !!locked; + }, + + // Call all callbacks with the given context and arguments + fireWith: function( context, args ) { + if ( !locked ) { + args = args || []; + args = [ context, args.slice ? args.slice() : args ]; + queue.push( args ); + if ( !firing ) { + fire(); + } + } + return this; + }, + + // Call all the callbacks with the given arguments + fire: function() { + self.fireWith( this, arguments ); + return this; + }, + + // To know if the callbacks have already been called at least once + fired: function() { + return !!fired; + } + }; + + return self; +}; + + +function Identity( v ) { + return v; +} +function Thrower( ex ) { + throw ex; +} + +function adoptValue( value, resolve, reject, noValue ) { + var method; + + try { + + // Check for promise aspect first to privilege synchronous behavior + if ( value && isFunction( ( method = value.promise ) ) ) { + method.call( value ).done( resolve ).fail( reject ); + + // Other thenables + } else if ( value && isFunction( ( method = value.then ) ) ) { + method.call( value, resolve, reject ); + + // Other non-thenables + } else { + + // Control `resolve` arguments by letting Array#slice cast boolean `noValue` to integer: + // * false: [ value ].slice( 0 ) => resolve( value ) + // * true: [ value ].slice( 1 ) => resolve() + resolve.apply( undefined, [ value ].slice( noValue ) ); + } + + // For Promises/A+, convert exceptions into rejections + // Since jQuery.when doesn't unwrap thenables, we can skip the extra checks appearing in + // Deferred#then to conditionally suppress rejection. + } catch ( value ) { + + // Support: Android 4.0 only + // Strict mode functions invoked without .call/.apply get global-object context + reject.apply( undefined, [ value ] ); + } +} + +jQuery.extend( { + + Deferred: function( func ) { + var tuples = [ + + // action, add listener, callbacks, + // ... .then handlers, argument index, [final state] + [ "notify", "progress", jQuery.Callbacks( "memory" ), + jQuery.Callbacks( "memory" ), 2 ], + [ "resolve", "done", jQuery.Callbacks( "once memory" ), + jQuery.Callbacks( "once memory" ), 0, "resolved" ], + [ "reject", "fail", jQuery.Callbacks( "once memory" ), + jQuery.Callbacks( "once memory" ), 1, "rejected" ] + ], + state = "pending", + promise = { + state: function() { + return state; + }, + always: function() { + deferred.done( arguments ).fail( arguments ); + return this; + }, + "catch": function( fn ) { + return promise.then( null, fn ); + }, + + // Keep pipe for back-compat + pipe: function( /* fnDone, fnFail, fnProgress */ ) { + var fns = arguments; + + return jQuery.Deferred( function( newDefer ) { + jQuery.each( tuples, function( i, tuple ) { + + // Map tuples (progress, done, fail) to arguments (done, fail, progress) + var fn = isFunction( fns[ tuple[ 4 ] ] ) && fns[ tuple[ 4 ] ]; + + // deferred.progress(function() { bind to newDefer or newDefer.notify }) + // deferred.done(function() { bind to newDefer or newDefer.resolve }) + // deferred.fail(function() { bind to newDefer or newDefer.reject }) + deferred[ tuple[ 1 ] ]( function() { + var returned = fn && fn.apply( this, arguments ); + if ( returned && isFunction( returned.promise ) ) { + returned.promise() + .progress( newDefer.notify ) + .done( newDefer.resolve ) + .fail( newDefer.reject ); + } else { + newDefer[ tuple[ 0 ] + "With" ]( + this, + fn ? [ returned ] : arguments + ); + } + } ); + } ); + fns = null; + } ).promise(); + }, + then: function( onFulfilled, onRejected, onProgress ) { + var maxDepth = 0; + function resolve( depth, deferred, handler, special ) { + return function() { + var that = this, + args = arguments, + mightThrow = function() { + var returned, then; + + // Support: Promises/A+ section 2.3.3.3.3 + // https://promisesaplus.com/#point-59 + // Ignore double-resolution attempts + if ( depth < maxDepth ) { + return; + } + + returned = handler.apply( that, args ); + + // Support: Promises/A+ section 2.3.1 + // https://promisesaplus.com/#point-48 + if ( returned === deferred.promise() ) { + throw new TypeError( "Thenable self-resolution" ); + } + + // Support: Promises/A+ sections 2.3.3.1, 3.5 + // https://promisesaplus.com/#point-54 + // https://promisesaplus.com/#point-75 + // Retrieve `then` only once + then = returned && + + // Support: Promises/A+ section 2.3.4 + // https://promisesaplus.com/#point-64 + // Only check objects and functions for thenability + ( typeof returned === "object" || + typeof returned === "function" ) && + returned.then; + + // Handle a returned thenable + if ( isFunction( then ) ) { + + // Special processors (notify) just wait for resolution + if ( special ) { + then.call( + returned, + resolve( maxDepth, deferred, Identity, special ), + resolve( maxDepth, deferred, Thrower, special ) + ); + + // Normal processors (resolve) also hook into progress + } else { + + // ...and disregard older resolution values + maxDepth++; + + then.call( + returned, + resolve( maxDepth, deferred, Identity, special ), + resolve( maxDepth, deferred, Thrower, special ), + resolve( maxDepth, deferred, Identity, + deferred.notifyWith ) + ); + } + + // Handle all other returned values + } else { + + // Only substitute handlers pass on context + // and multiple values (non-spec behavior) + if ( handler !== Identity ) { + that = undefined; + args = [ returned ]; + } + + // Process the value(s) + // Default process is resolve + ( special || deferred.resolveWith )( that, args ); + } + }, + + // Only normal processors (resolve) catch and reject exceptions + process = special ? + mightThrow : + function() { + try { + mightThrow(); + } catch ( e ) { + + if ( jQuery.Deferred.exceptionHook ) { + jQuery.Deferred.exceptionHook( e, + process.stackTrace ); + } + + // Support: Promises/A+ section 2.3.3.3.4.1 + // https://promisesaplus.com/#point-61 + // Ignore post-resolution exceptions + if ( depth + 1 >= maxDepth ) { + + // Only substitute handlers pass on context + // and multiple values (non-spec behavior) + if ( handler !== Thrower ) { + that = undefined; + args = [ e ]; + } + + deferred.rejectWith( that, args ); + } + } + }; + + // Support: Promises/A+ section 2.3.3.3.1 + // https://promisesaplus.com/#point-57 + // Re-resolve promises immediately to dodge false rejection from + // subsequent errors + if ( depth ) { + process(); + } else { + + // Call an optional hook to record the stack, in case of exception + // since it's otherwise lost when execution goes async + if ( jQuery.Deferred.getStackHook ) { + process.stackTrace = jQuery.Deferred.getStackHook(); + } + window.setTimeout( process ); + } + }; + } + + return jQuery.Deferred( function( newDefer ) { + + // progress_handlers.add( ... ) + tuples[ 0 ][ 3 ].add( + resolve( + 0, + newDefer, + isFunction( onProgress ) ? + onProgress : + Identity, + newDefer.notifyWith + ) + ); + + // fulfilled_handlers.add( ... ) + tuples[ 1 ][ 3 ].add( + resolve( + 0, + newDefer, + isFunction( onFulfilled ) ? + onFulfilled : + Identity + ) + ); + + // rejected_handlers.add( ... ) + tuples[ 2 ][ 3 ].add( + resolve( + 0, + newDefer, + isFunction( onRejected ) ? + onRejected : + Thrower + ) + ); + } ).promise(); + }, + + // Get a promise for this deferred + // If obj is provided, the promise aspect is added to the object + promise: function( obj ) { + return obj != null ? jQuery.extend( obj, promise ) : promise; + } + }, + deferred = {}; + + // Add list-specific methods + jQuery.each( tuples, function( i, tuple ) { + var list = tuple[ 2 ], + stateString = tuple[ 5 ]; + + // promise.progress = list.add + // promise.done = list.add + // promise.fail = list.add + promise[ tuple[ 1 ] ] = list.add; + + // Handle state + if ( stateString ) { + list.add( + function() { + + // state = "resolved" (i.e., fulfilled) + // state = "rejected" + state = stateString; + }, + + // rejected_callbacks.disable + // fulfilled_callbacks.disable + tuples[ 3 - i ][ 2 ].disable, + + // rejected_handlers.disable + // fulfilled_handlers.disable + tuples[ 3 - i ][ 3 ].disable, + + // progress_callbacks.lock + tuples[ 0 ][ 2 ].lock, + + // progress_handlers.lock + tuples[ 0 ][ 3 ].lock + ); + } + + // progress_handlers.fire + // fulfilled_handlers.fire + // rejected_handlers.fire + list.add( tuple[ 3 ].fire ); + + // deferred.notify = function() { deferred.notifyWith(...) } + // deferred.resolve = function() { deferred.resolveWith(...) } + // deferred.reject = function() { deferred.rejectWith(...) } + deferred[ tuple[ 0 ] ] = function() { + deferred[ tuple[ 0 ] + "With" ]( this === deferred ? undefined : this, arguments ); + return this; + }; + + // deferred.notifyWith = list.fireWith + // deferred.resolveWith = list.fireWith + // deferred.rejectWith = list.fireWith + deferred[ tuple[ 0 ] + "With" ] = list.fireWith; + } ); + + // Make the deferred a promise + promise.promise( deferred ); + + // Call given func if any + if ( func ) { + func.call( deferred, deferred ); + } + + // All done! + return deferred; + }, + + // Deferred helper + when: function( singleValue ) { + var + + // count of uncompleted subordinates + remaining = arguments.length, + + // count of unprocessed arguments + i = remaining, + + // subordinate fulfillment data + resolveContexts = Array( i ), + resolveValues = slice.call( arguments ), + + // the master Deferred + master = jQuery.Deferred(), + + // subordinate callback factory + updateFunc = function( i ) { + return function( value ) { + resolveContexts[ i ] = this; + resolveValues[ i ] = arguments.length > 1 ? slice.call( arguments ) : value; + if ( !( --remaining ) ) { + master.resolveWith( resolveContexts, resolveValues ); + } + }; + }; + + // Single- and empty arguments are adopted like Promise.resolve + if ( remaining <= 1 ) { + adoptValue( singleValue, master.done( updateFunc( i ) ).resolve, master.reject, + !remaining ); + + // Use .then() to unwrap secondary thenables (cf. gh-3000) + if ( master.state() === "pending" || + isFunction( resolveValues[ i ] && resolveValues[ i ].then ) ) { + + return master.then(); + } + } + + // Multiple arguments are aggregated like Promise.all array elements + while ( i-- ) { + adoptValue( resolveValues[ i ], updateFunc( i ), master.reject ); + } + + return master.promise(); + } +} ); + + +// These usually indicate a programmer mistake during development, +// warn about them ASAP rather than swallowing them by default. +var rerrorNames = /^(Eval|Internal|Range|Reference|Syntax|Type|URI)Error$/; + +jQuery.Deferred.exceptionHook = function( error, stack ) { + + // Support: IE 8 - 9 only + // Console exists when dev tools are open, which can happen at any time + if ( window.console && window.console.warn && error && rerrorNames.test( error.name ) ) { + window.console.warn( "jQuery.Deferred exception: " + error.message, error.stack, stack ); + } +}; + + + + +jQuery.readyException = function( error ) { + window.setTimeout( function() { + throw error; + } ); +}; + + + + +// The deferred used on DOM ready +var readyList = jQuery.Deferred(); + +jQuery.fn.ready = function( fn ) { + + readyList + .then( fn ) + + // Wrap jQuery.readyException in a function so that the lookup + // happens at the time of error handling instead of callback + // registration. + .catch( function( error ) { + jQuery.readyException( error ); + } ); + + return this; +}; + +jQuery.extend( { + + // Is the DOM ready to be used? Set to true once it occurs. + isReady: false, + + // A counter to track how many items to wait for before + // the ready event fires. See #6781 + readyWait: 1, + + // Handle when the DOM is ready + ready: function( wait ) { + + // Abort if there are pending holds or we're already ready + if ( wait === true ? --jQuery.readyWait : jQuery.isReady ) { + return; + } + + // Remember that the DOM is ready + jQuery.isReady = true; + + // If a normal DOM Ready event fired, decrement, and wait if need be + if ( wait !== true && --jQuery.readyWait > 0 ) { + return; + } + + // If there are functions bound, to execute + readyList.resolveWith( document, [ jQuery ] ); + } +} ); + +jQuery.ready.then = readyList.then; + +// The ready event handler and self cleanup method +function completed() { + document.removeEventListener( "DOMContentLoaded", completed ); + window.removeEventListener( "load", completed ); + jQuery.ready(); +} + +// Catch cases where $(document).ready() is called +// after the browser event has already occurred. +// Support: IE <=9 - 10 only +// Older IE sometimes signals "interactive" too soon +if ( document.readyState === "complete" || + ( document.readyState !== "loading" && !document.documentElement.doScroll ) ) { + + // Handle it asynchronously to allow scripts the opportunity to delay ready + window.setTimeout( jQuery.ready ); + +} else { + + // Use the handy event callback + document.addEventListener( "DOMContentLoaded", completed ); + + // A fallback to window.onload, that will always work + window.addEventListener( "load", completed ); +} + + + + +// Multifunctional method to get and set values of a collection +// The value/s can optionally be executed if it's a function +var access = function( elems, fn, key, value, chainable, emptyGet, raw ) { + var i = 0, + len = elems.length, + bulk = key == null; + + // Sets many values + if ( toType( key ) === "object" ) { + chainable = true; + for ( i in key ) { + access( elems, fn, i, key[ i ], true, emptyGet, raw ); + } + + // Sets one value + } else if ( value !== undefined ) { + chainable = true; + + if ( !isFunction( value ) ) { + raw = true; + } + + if ( bulk ) { + + // Bulk operations run against the entire set + if ( raw ) { + fn.call( elems, value ); + fn = null; + + // ...except when executing function values + } else { + bulk = fn; + fn = function( elem, key, value ) { + return bulk.call( jQuery( elem ), value ); + }; + } + } + + if ( fn ) { + for ( ; i < len; i++ ) { + fn( + elems[ i ], key, raw ? + value : + value.call( elems[ i ], i, fn( elems[ i ], key ) ) + ); + } + } + } + + if ( chainable ) { + return elems; + } + + // Gets + if ( bulk ) { + return fn.call( elems ); + } + + return len ? fn( elems[ 0 ], key ) : emptyGet; +}; + + +// Matches dashed string for camelizing +var rmsPrefix = /^-ms-/, + rdashAlpha = /-([a-z])/g; + +// Used by camelCase as callback to replace() +function fcamelCase( all, letter ) { + return letter.toUpperCase(); +} + +// Convert dashed to camelCase; used by the css and data modules +// Support: IE <=9 - 11, Edge 12 - 15 +// Microsoft forgot to hump their vendor prefix (#9572) +function camelCase( string ) { + return string.replace( rmsPrefix, "ms-" ).replace( rdashAlpha, fcamelCase ); +} +var acceptData = function( owner ) { + + // Accepts only: + // - Node + // - Node.ELEMENT_NODE + // - Node.DOCUMENT_NODE + // - Object + // - Any + return owner.nodeType === 1 || owner.nodeType === 9 || !( +owner.nodeType ); +}; + + + + +function Data() { + this.expando = jQuery.expando + Data.uid++; +} + +Data.uid = 1; + +Data.prototype = { + + cache: function( owner ) { + + // Check if the owner object already has a cache + var value = owner[ this.expando ]; + + // If not, create one + if ( !value ) { + value = {}; + + // We can accept data for non-element nodes in modern browsers, + // but we should not, see #8335. + // Always return an empty object. + if ( acceptData( owner ) ) { + + // If it is a node unlikely to be stringify-ed or looped over + // use plain assignment + if ( owner.nodeType ) { + owner[ this.expando ] = value; + + // Otherwise secure it in a non-enumerable property + // configurable must be true to allow the property to be + // deleted when data is removed + } else { + Object.defineProperty( owner, this.expando, { + value: value, + configurable: true + } ); + } + } + } + + return value; + }, + set: function( owner, data, value ) { + var prop, + cache = this.cache( owner ); + + // Handle: [ owner, key, value ] args + // Always use camelCase key (gh-2257) + if ( typeof data === "string" ) { + cache[ camelCase( data ) ] = value; + + // Handle: [ owner, { properties } ] args + } else { + + // Copy the properties one-by-one to the cache object + for ( prop in data ) { + cache[ camelCase( prop ) ] = data[ prop ]; + } + } + return cache; + }, + get: function( owner, key ) { + return key === undefined ? + this.cache( owner ) : + + // Always use camelCase key (gh-2257) + owner[ this.expando ] && owner[ this.expando ][ camelCase( key ) ]; + }, + access: function( owner, key, value ) { + + // In cases where either: + // + // 1. No key was specified + // 2. A string key was specified, but no value provided + // + // Take the "read" path and allow the get method to determine + // which value to return, respectively either: + // + // 1. The entire cache object + // 2. The data stored at the key + // + if ( key === undefined || + ( ( key && typeof key === "string" ) && value === undefined ) ) { + + return this.get( owner, key ); + } + + // When the key is not a string, or both a key and value + // are specified, set or extend (existing objects) with either: + // + // 1. An object of properties + // 2. A key and value + // + this.set( owner, key, value ); + + // Since the "set" path can have two possible entry points + // return the expected data based on which path was taken[*] + return value !== undefined ? value : key; + }, + remove: function( owner, key ) { + var i, + cache = owner[ this.expando ]; + + if ( cache === undefined ) { + return; + } + + if ( key !== undefined ) { + + // Support array or space separated string of keys + if ( Array.isArray( key ) ) { + + // If key is an array of keys... + // We always set camelCase keys, so remove that. + key = key.map( camelCase ); + } else { + key = camelCase( key ); + + // If a key with the spaces exists, use it. + // Otherwise, create an array by matching non-whitespace + key = key in cache ? + [ key ] : + ( key.match( rnothtmlwhite ) || [] ); + } + + i = key.length; + + while ( i-- ) { + delete cache[ key[ i ] ]; + } + } + + // Remove the expando if there's no more data + if ( key === undefined || jQuery.isEmptyObject( cache ) ) { + + // Support: Chrome <=35 - 45 + // Webkit & Blink performance suffers when deleting properties + // from DOM nodes, so set to undefined instead + // https://bugs.chromium.org/p/chromium/issues/detail?id=378607 (bug restricted) + if ( owner.nodeType ) { + owner[ this.expando ] = undefined; + } else { + delete owner[ this.expando ]; + } + } + }, + hasData: function( owner ) { + var cache = owner[ this.expando ]; + return cache !== undefined && !jQuery.isEmptyObject( cache ); + } +}; +var dataPriv = new Data(); + +var dataUser = new Data(); + + + +// Implementation Summary +// +// 1. Enforce API surface and semantic compatibility with 1.9.x branch +// 2. Improve the module's maintainability by reducing the storage +// paths to a single mechanism. +// 3. Use the same single mechanism to support "private" and "user" data. +// 4. _Never_ expose "private" data to user code (TODO: Drop _data, _removeData) +// 5. Avoid exposing implementation details on user objects (eg. expando properties) +// 6. Provide a clear path for implementation upgrade to WeakMap in 2014 + +var rbrace = /^(?:\{[\w\W]*\}|\[[\w\W]*\])$/, + rmultiDash = /[A-Z]/g; + +function getData( data ) { + if ( data === "true" ) { + return true; + } + + if ( data === "false" ) { + return false; + } + + if ( data === "null" ) { + return null; + } + + // Only convert to a number if it doesn't change the string + if ( data === +data + "" ) { + return +data; + } + + if ( rbrace.test( data ) ) { + return JSON.parse( data ); + } + + return data; +} + +function dataAttr( elem, key, data ) { + var name; + + // If nothing was found internally, try to fetch any + // data from the HTML5 data-* attribute + if ( data === undefined && elem.nodeType === 1 ) { + name = "data-" + key.replace( rmultiDash, "-$&" ).toLowerCase(); + data = elem.getAttribute( name ); + + if ( typeof data === "string" ) { + try { + data = getData( data ); + } catch ( e ) {} + + // Make sure we set the data so it isn't changed later + dataUser.set( elem, key, data ); + } else { + data = undefined; + } + } + return data; +} + +jQuery.extend( { + hasData: function( elem ) { + return dataUser.hasData( elem ) || dataPriv.hasData( elem ); + }, + + data: function( elem, name, data ) { + return dataUser.access( elem, name, data ); + }, + + removeData: function( elem, name ) { + dataUser.remove( elem, name ); + }, + + // TODO: Now that all calls to _data and _removeData have been replaced + // with direct calls to dataPriv methods, these can be deprecated. + _data: function( elem, name, data ) { + return dataPriv.access( elem, name, data ); + }, + + _removeData: function( elem, name ) { + dataPriv.remove( elem, name ); + } +} ); + +jQuery.fn.extend( { + data: function( key, value ) { + var i, name, data, + elem = this[ 0 ], + attrs = elem && elem.attributes; + + // Gets all values + if ( key === undefined ) { + if ( this.length ) { + data = dataUser.get( elem ); + + if ( elem.nodeType === 1 && !dataPriv.get( elem, "hasDataAttrs" ) ) { + i = attrs.length; + while ( i-- ) { + + // Support: IE 11 only + // The attrs elements can be null (#14894) + if ( attrs[ i ] ) { + name = attrs[ i ].name; + if ( name.indexOf( "data-" ) === 0 ) { + name = camelCase( name.slice( 5 ) ); + dataAttr( elem, name, data[ name ] ); + } + } + } + dataPriv.set( elem, "hasDataAttrs", true ); + } + } + + return data; + } + + // Sets multiple values + if ( typeof key === "object" ) { + return this.each( function() { + dataUser.set( this, key ); + } ); + } + + return access( this, function( value ) { + var data; + + // The calling jQuery object (element matches) is not empty + // (and therefore has an element appears at this[ 0 ]) and the + // `value` parameter was not undefined. An empty jQuery object + // will result in `undefined` for elem = this[ 0 ] which will + // throw an exception if an attempt to read a data cache is made. + if ( elem && value === undefined ) { + + // Attempt to get data from the cache + // The key will always be camelCased in Data + data = dataUser.get( elem, key ); + if ( data !== undefined ) { + return data; + } + + // Attempt to "discover" the data in + // HTML5 custom data-* attrs + data = dataAttr( elem, key ); + if ( data !== undefined ) { + return data; + } + + // We tried really hard, but the data doesn't exist. + return; + } + + // Set the data... + this.each( function() { + + // We always store the camelCased key + dataUser.set( this, key, value ); + } ); + }, null, value, arguments.length > 1, null, true ); + }, + + removeData: function( key ) { + return this.each( function() { + dataUser.remove( this, key ); + } ); + } +} ); + + +jQuery.extend( { + queue: function( elem, type, data ) { + var queue; + + if ( elem ) { + type = ( type || "fx" ) + "queue"; + queue = dataPriv.get( elem, type ); + + // Speed up dequeue by getting out quickly if this is just a lookup + if ( data ) { + if ( !queue || Array.isArray( data ) ) { + queue = dataPriv.access( elem, type, jQuery.makeArray( data ) ); + } else { + queue.push( data ); + } + } + return queue || []; + } + }, + + dequeue: function( elem, type ) { + type = type || "fx"; + + var queue = jQuery.queue( elem, type ), + startLength = queue.length, + fn = queue.shift(), + hooks = jQuery._queueHooks( elem, type ), + next = function() { + jQuery.dequeue( elem, type ); + }; + + // If the fx queue is dequeued, always remove the progress sentinel + if ( fn === "inprogress" ) { + fn = queue.shift(); + startLength--; + } + + if ( fn ) { + + // Add a progress sentinel to prevent the fx queue from being + // automatically dequeued + if ( type === "fx" ) { + queue.unshift( "inprogress" ); + } + + // Clear up the last queue stop function + delete hooks.stop; + fn.call( elem, next, hooks ); + } + + if ( !startLength && hooks ) { + hooks.empty.fire(); + } + }, + + // Not public - generate a queueHooks object, or return the current one + _queueHooks: function( elem, type ) { + var key = type + "queueHooks"; + return dataPriv.get( elem, key ) || dataPriv.access( elem, key, { + empty: jQuery.Callbacks( "once memory" ).add( function() { + dataPriv.remove( elem, [ type + "queue", key ] ); + } ) + } ); + } +} ); + +jQuery.fn.extend( { + queue: function( type, data ) { + var setter = 2; + + if ( typeof type !== "string" ) { + data = type; + type = "fx"; + setter--; + } + + if ( arguments.length < setter ) { + return jQuery.queue( this[ 0 ], type ); + } + + return data === undefined ? + this : + this.each( function() { + var queue = jQuery.queue( this, type, data ); + + // Ensure a hooks for this queue + jQuery._queueHooks( this, type ); + + if ( type === "fx" && queue[ 0 ] !== "inprogress" ) { + jQuery.dequeue( this, type ); + } + } ); + }, + dequeue: function( type ) { + return this.each( function() { + jQuery.dequeue( this, type ); + } ); + }, + clearQueue: function( type ) { + return this.queue( type || "fx", [] ); + }, + + // Get a promise resolved when queues of a certain type + // are emptied (fx is the type by default) + promise: function( type, obj ) { + var tmp, + count = 1, + defer = jQuery.Deferred(), + elements = this, + i = this.length, + resolve = function() { + if ( !( --count ) ) { + defer.resolveWith( elements, [ elements ] ); + } + }; + + if ( typeof type !== "string" ) { + obj = type; + type = undefined; + } + type = type || "fx"; + + while ( i-- ) { + tmp = dataPriv.get( elements[ i ], type + "queueHooks" ); + if ( tmp && tmp.empty ) { + count++; + tmp.empty.add( resolve ); + } + } + resolve(); + return defer.promise( obj ); + } +} ); +var pnum = ( /[+-]?(?:\d*\.|)\d+(?:[eE][+-]?\d+|)/ ).source; + +var rcssNum = new RegExp( "^(?:([+-])=|)(" + pnum + ")([a-z%]*)$", "i" ); + + +var cssExpand = [ "Top", "Right", "Bottom", "Left" ]; + +var documentElement = document.documentElement; + + + + var isAttached = function( elem ) { + return jQuery.contains( elem.ownerDocument, elem ); + }, + composed = { composed: true }; + + // Support: IE 9 - 11+, Edge 12 - 18+, iOS 10.0 - 10.2 only + // Check attachment across shadow DOM boundaries when possible (gh-3504) + // Support: iOS 10.0-10.2 only + // Early iOS 10 versions support `attachShadow` but not `getRootNode`, + // leading to errors. We need to check for `getRootNode`. + if ( documentElement.getRootNode ) { + isAttached = function( elem ) { + return jQuery.contains( elem.ownerDocument, elem ) || + elem.getRootNode( composed ) === elem.ownerDocument; + }; + } +var isHiddenWithinTree = function( elem, el ) { + + // isHiddenWithinTree might be called from jQuery#filter function; + // in that case, element will be second argument + elem = el || elem; + + // Inline style trumps all + return elem.style.display === "none" || + elem.style.display === "" && + + // Otherwise, check computed style + // Support: Firefox <=43 - 45 + // Disconnected elements can have computed display: none, so first confirm that elem is + // in the document. + isAttached( elem ) && + + jQuery.css( elem, "display" ) === "none"; + }; + +var swap = function( elem, options, callback, args ) { + var ret, name, + old = {}; + + // Remember the old values, and insert the new ones + for ( name in options ) { + old[ name ] = elem.style[ name ]; + elem.style[ name ] = options[ name ]; + } + + ret = callback.apply( elem, args || [] ); + + // Revert the old values + for ( name in options ) { + elem.style[ name ] = old[ name ]; + } + + return ret; +}; + + + + +function adjustCSS( elem, prop, valueParts, tween ) { + var adjusted, scale, + maxIterations = 20, + currentValue = tween ? + function() { + return tween.cur(); + } : + function() { + return jQuery.css( elem, prop, "" ); + }, + initial = currentValue(), + unit = valueParts && valueParts[ 3 ] || ( jQuery.cssNumber[ prop ] ? "" : "px" ), + + // Starting value computation is required for potential unit mismatches + initialInUnit = elem.nodeType && + ( jQuery.cssNumber[ prop ] || unit !== "px" && +initial ) && + rcssNum.exec( jQuery.css( elem, prop ) ); + + if ( initialInUnit && initialInUnit[ 3 ] !== unit ) { + + // Support: Firefox <=54 + // Halve the iteration target value to prevent interference from CSS upper bounds (gh-2144) + initial = initial / 2; + + // Trust units reported by jQuery.css + unit = unit || initialInUnit[ 3 ]; + + // Iteratively approximate from a nonzero starting point + initialInUnit = +initial || 1; + + while ( maxIterations-- ) { + + // Evaluate and update our best guess (doubling guesses that zero out). + // Finish if the scale equals or crosses 1 (making the old*new product non-positive). + jQuery.style( elem, prop, initialInUnit + unit ); + if ( ( 1 - scale ) * ( 1 - ( scale = currentValue() / initial || 0.5 ) ) <= 0 ) { + maxIterations = 0; + } + initialInUnit = initialInUnit / scale; + + } + + initialInUnit = initialInUnit * 2; + jQuery.style( elem, prop, initialInUnit + unit ); + + // Make sure we update the tween properties later on + valueParts = valueParts || []; + } + + if ( valueParts ) { + initialInUnit = +initialInUnit || +initial || 0; + + // Apply relative offset (+=/-=) if specified + adjusted = valueParts[ 1 ] ? + initialInUnit + ( valueParts[ 1 ] + 1 ) * valueParts[ 2 ] : + +valueParts[ 2 ]; + if ( tween ) { + tween.unit = unit; + tween.start = initialInUnit; + tween.end = adjusted; + } + } + return adjusted; +} + + +var defaultDisplayMap = {}; + +function getDefaultDisplay( elem ) { + var temp, + doc = elem.ownerDocument, + nodeName = elem.nodeName, + display = defaultDisplayMap[ nodeName ]; + + if ( display ) { + return display; + } + + temp = doc.body.appendChild( doc.createElement( nodeName ) ); + display = jQuery.css( temp, "display" ); + + temp.parentNode.removeChild( temp ); + + if ( display === "none" ) { + display = "block"; + } + defaultDisplayMap[ nodeName ] = display; + + return display; +} + +function showHide( elements, show ) { + var display, elem, + values = [], + index = 0, + length = elements.length; + + // Determine new display value for elements that need to change + for ( ; index < length; index++ ) { + elem = elements[ index ]; + if ( !elem.style ) { + continue; + } + + display = elem.style.display; + if ( show ) { + + // Since we force visibility upon cascade-hidden elements, an immediate (and slow) + // check is required in this first loop unless we have a nonempty display value (either + // inline or about-to-be-restored) + if ( display === "none" ) { + values[ index ] = dataPriv.get( elem, "display" ) || null; + if ( !values[ index ] ) { + elem.style.display = ""; + } + } + if ( elem.style.display === "" && isHiddenWithinTree( elem ) ) { + values[ index ] = getDefaultDisplay( elem ); + } + } else { + if ( display !== "none" ) { + values[ index ] = "none"; + + // Remember what we're overwriting + dataPriv.set( elem, "display", display ); + } + } + } + + // Set the display of the elements in a second loop to avoid constant reflow + for ( index = 0; index < length; index++ ) { + if ( values[ index ] != null ) { + elements[ index ].style.display = values[ index ]; + } + } + + return elements; +} + +jQuery.fn.extend( { + show: function() { + return showHide( this, true ); + }, + hide: function() { + return showHide( this ); + }, + toggle: function( state ) { + if ( typeof state === "boolean" ) { + return state ? this.show() : this.hide(); + } + + return this.each( function() { + if ( isHiddenWithinTree( this ) ) { + jQuery( this ).show(); + } else { + jQuery( this ).hide(); + } + } ); + } +} ); +var rcheckableType = ( /^(?:checkbox|radio)$/i ); + +var rtagName = ( /<([a-z][^\/\0>\x20\t\r\n\f]*)/i ); + +var rscriptType = ( /^$|^module$|\/(?:java|ecma)script/i ); + + + +// We have to close these tags to support XHTML (#13200) +var wrapMap = { + + // Support: IE <=9 only + option: [ 1, "" ], + + // XHTML parsers do not magically insert elements in the + // same way that tag soup parsers do. So we cannot shorten + // this by omitting or other required elements. + thead: [ 1, "", "
" ], + col: [ 2, "", "
" ], + tr: [ 2, "", "
" ], + td: [ 3, "", "
" ], + + _default: [ 0, "", "" ] +}; + +// Support: IE <=9 only +wrapMap.optgroup = wrapMap.option; + +wrapMap.tbody = wrapMap.tfoot = wrapMap.colgroup = wrapMap.caption = wrapMap.thead; +wrapMap.th = wrapMap.td; + + +function getAll( context, tag ) { + + // Support: IE <=9 - 11 only + // Use typeof to avoid zero-argument method invocation on host objects (#15151) + var ret; + + if ( typeof context.getElementsByTagName !== "undefined" ) { + ret = context.getElementsByTagName( tag || "*" ); + + } else if ( typeof context.querySelectorAll !== "undefined" ) { + ret = context.querySelectorAll( tag || "*" ); + + } else { + ret = []; + } + + if ( tag === undefined || tag && nodeName( context, tag ) ) { + return jQuery.merge( [ context ], ret ); + } + + return ret; +} + + +// Mark scripts as having already been evaluated +function setGlobalEval( elems, refElements ) { + var i = 0, + l = elems.length; + + for ( ; i < l; i++ ) { + dataPriv.set( + elems[ i ], + "globalEval", + !refElements || dataPriv.get( refElements[ i ], "globalEval" ) + ); + } +} + + +var rhtml = /<|&#?\w+;/; + +function buildFragment( elems, context, scripts, selection, ignored ) { + var elem, tmp, tag, wrap, attached, j, + fragment = context.createDocumentFragment(), + nodes = [], + i = 0, + l = elems.length; + + for ( ; i < l; i++ ) { + elem = elems[ i ]; + + if ( elem || elem === 0 ) { + + // Add nodes directly + if ( toType( elem ) === "object" ) { + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + jQuery.merge( nodes, elem.nodeType ? [ elem ] : elem ); + + // Convert non-html into a text node + } else if ( !rhtml.test( elem ) ) { + nodes.push( context.createTextNode( elem ) ); + + // Convert html into DOM nodes + } else { + tmp = tmp || fragment.appendChild( context.createElement( "div" ) ); + + // Deserialize a standard representation + tag = ( rtagName.exec( elem ) || [ "", "" ] )[ 1 ].toLowerCase(); + wrap = wrapMap[ tag ] || wrapMap._default; + tmp.innerHTML = wrap[ 1 ] + jQuery.htmlPrefilter( elem ) + wrap[ 2 ]; + + // Descend through wrappers to the right content + j = wrap[ 0 ]; + while ( j-- ) { + tmp = tmp.lastChild; + } + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + jQuery.merge( nodes, tmp.childNodes ); + + // Remember the top-level container + tmp = fragment.firstChild; + + // Ensure the created nodes are orphaned (#12392) + tmp.textContent = ""; + } + } + } + + // Remove wrapper from fragment + fragment.textContent = ""; + + i = 0; + while ( ( elem = nodes[ i++ ] ) ) { + + // Skip elements already in the context collection (trac-4087) + if ( selection && jQuery.inArray( elem, selection ) > -1 ) { + if ( ignored ) { + ignored.push( elem ); + } + continue; + } + + attached = isAttached( elem ); + + // Append to fragment + tmp = getAll( fragment.appendChild( elem ), "script" ); + + // Preserve script evaluation history + if ( attached ) { + setGlobalEval( tmp ); + } + + // Capture executables + if ( scripts ) { + j = 0; + while ( ( elem = tmp[ j++ ] ) ) { + if ( rscriptType.test( elem.type || "" ) ) { + scripts.push( elem ); + } + } + } + } + + return fragment; +} + + +( function() { + var fragment = document.createDocumentFragment(), + div = fragment.appendChild( document.createElement( "div" ) ), + input = document.createElement( "input" ); + + // Support: Android 4.0 - 4.3 only + // Check state lost if the name is set (#11217) + // Support: Windows Web Apps (WWA) + // `name` and `type` must use .setAttribute for WWA (#14901) + input.setAttribute( "type", "radio" ); + input.setAttribute( "checked", "checked" ); + input.setAttribute( "name", "t" ); + + div.appendChild( input ); + + // Support: Android <=4.1 only + // Older WebKit doesn't clone checked state correctly in fragments + support.checkClone = div.cloneNode( true ).cloneNode( true ).lastChild.checked; + + // Support: IE <=11 only + // Make sure textarea (and checkbox) defaultValue is properly cloned + div.innerHTML = ""; + support.noCloneChecked = !!div.cloneNode( true ).lastChild.defaultValue; +} )(); + + +var + rkeyEvent = /^key/, + rmouseEvent = /^(?:mouse|pointer|contextmenu|drag|drop)|click/, + rtypenamespace = /^([^.]*)(?:\.(.+)|)/; + +function returnTrue() { + return true; +} + +function returnFalse() { + return false; +} + +// Support: IE <=9 - 11+ +// focus() and blur() are asynchronous, except when they are no-op. +// So expect focus to be synchronous when the element is already active, +// and blur to be synchronous when the element is not already active. +// (focus and blur are always synchronous in other supported browsers, +// this just defines when we can count on it). +function expectSync( elem, type ) { + return ( elem === safeActiveElement() ) === ( type === "focus" ); +} + +// Support: IE <=9 only +// Accessing document.activeElement can throw unexpectedly +// https://bugs.jquery.com/ticket/13393 +function safeActiveElement() { + try { + return document.activeElement; + } catch ( err ) { } +} + +function on( elem, types, selector, data, fn, one ) { + var origFn, type; + + // Types can be a map of types/handlers + if ( typeof types === "object" ) { + + // ( types-Object, selector, data ) + if ( typeof selector !== "string" ) { + + // ( types-Object, data ) + data = data || selector; + selector = undefined; + } + for ( type in types ) { + on( elem, type, selector, data, types[ type ], one ); + } + return elem; + } + + if ( data == null && fn == null ) { + + // ( types, fn ) + fn = selector; + data = selector = undefined; + } else if ( fn == null ) { + if ( typeof selector === "string" ) { + + // ( types, selector, fn ) + fn = data; + data = undefined; + } else { + + // ( types, data, fn ) + fn = data; + data = selector; + selector = undefined; + } + } + if ( fn === false ) { + fn = returnFalse; + } else if ( !fn ) { + return elem; + } + + if ( one === 1 ) { + origFn = fn; + fn = function( event ) { + + // Can use an empty set, since event contains the info + jQuery().off( event ); + return origFn.apply( this, arguments ); + }; + + // Use same guid so caller can remove using origFn + fn.guid = origFn.guid || ( origFn.guid = jQuery.guid++ ); + } + return elem.each( function() { + jQuery.event.add( this, types, fn, data, selector ); + } ); +} + +/* + * Helper functions for managing events -- not part of the public interface. + * Props to Dean Edwards' addEvent library for many of the ideas. + */ +jQuery.event = { + + global: {}, + + add: function( elem, types, handler, data, selector ) { + + var handleObjIn, eventHandle, tmp, + events, t, handleObj, + special, handlers, type, namespaces, origType, + elemData = dataPriv.get( elem ); + + // Don't attach events to noData or text/comment nodes (but allow plain objects) + if ( !elemData ) { + return; + } + + // Caller can pass in an object of custom data in lieu of the handler + if ( handler.handler ) { + handleObjIn = handler; + handler = handleObjIn.handler; + selector = handleObjIn.selector; + } + + // Ensure that invalid selectors throw exceptions at attach time + // Evaluate against documentElement in case elem is a non-element node (e.g., document) + if ( selector ) { + jQuery.find.matchesSelector( documentElement, selector ); + } + + // Make sure that the handler has a unique ID, used to find/remove it later + if ( !handler.guid ) { + handler.guid = jQuery.guid++; + } + + // Init the element's event structure and main handler, if this is the first + if ( !( events = elemData.events ) ) { + events = elemData.events = {}; + } + if ( !( eventHandle = elemData.handle ) ) { + eventHandle = elemData.handle = function( e ) { + + // Discard the second event of a jQuery.event.trigger() and + // when an event is called after a page has unloaded + return typeof jQuery !== "undefined" && jQuery.event.triggered !== e.type ? + jQuery.event.dispatch.apply( elem, arguments ) : undefined; + }; + } + + // Handle multiple events separated by a space + types = ( types || "" ).match( rnothtmlwhite ) || [ "" ]; + t = types.length; + while ( t-- ) { + tmp = rtypenamespace.exec( types[ t ] ) || []; + type = origType = tmp[ 1 ]; + namespaces = ( tmp[ 2 ] || "" ).split( "." ).sort(); + + // There *must* be a type, no attaching namespace-only handlers + if ( !type ) { + continue; + } + + // If event changes its type, use the special event handlers for the changed type + special = jQuery.event.special[ type ] || {}; + + // If selector defined, determine special event api type, otherwise given type + type = ( selector ? special.delegateType : special.bindType ) || type; + + // Update special based on newly reset type + special = jQuery.event.special[ type ] || {}; + + // handleObj is passed to all event handlers + handleObj = jQuery.extend( { + type: type, + origType: origType, + data: data, + handler: handler, + guid: handler.guid, + selector: selector, + needsContext: selector && jQuery.expr.match.needsContext.test( selector ), + namespace: namespaces.join( "." ) + }, handleObjIn ); + + // Init the event handler queue if we're the first + if ( !( handlers = events[ type ] ) ) { + handlers = events[ type ] = []; + handlers.delegateCount = 0; + + // Only use addEventListener if the special events handler returns false + if ( !special.setup || + special.setup.call( elem, data, namespaces, eventHandle ) === false ) { + + if ( elem.addEventListener ) { + elem.addEventListener( type, eventHandle ); + } + } + } + + if ( special.add ) { + special.add.call( elem, handleObj ); + + if ( !handleObj.handler.guid ) { + handleObj.handler.guid = handler.guid; + } + } + + // Add to the element's handler list, delegates in front + if ( selector ) { + handlers.splice( handlers.delegateCount++, 0, handleObj ); + } else { + handlers.push( handleObj ); + } + + // Keep track of which events have ever been used, for event optimization + jQuery.event.global[ type ] = true; + } + + }, + + // Detach an event or set of events from an element + remove: function( elem, types, handler, selector, mappedTypes ) { + + var j, origCount, tmp, + events, t, handleObj, + special, handlers, type, namespaces, origType, + elemData = dataPriv.hasData( elem ) && dataPriv.get( elem ); + + if ( !elemData || !( events = elemData.events ) ) { + return; + } + + // Once for each type.namespace in types; type may be omitted + types = ( types || "" ).match( rnothtmlwhite ) || [ "" ]; + t = types.length; + while ( t-- ) { + tmp = rtypenamespace.exec( types[ t ] ) || []; + type = origType = tmp[ 1 ]; + namespaces = ( tmp[ 2 ] || "" ).split( "." ).sort(); + + // Unbind all events (on this namespace, if provided) for the element + if ( !type ) { + for ( type in events ) { + jQuery.event.remove( elem, type + types[ t ], handler, selector, true ); + } + continue; + } + + special = jQuery.event.special[ type ] || {}; + type = ( selector ? special.delegateType : special.bindType ) || type; + handlers = events[ type ] || []; + tmp = tmp[ 2 ] && + new RegExp( "(^|\\.)" + namespaces.join( "\\.(?:.*\\.|)" ) + "(\\.|$)" ); + + // Remove matching events + origCount = j = handlers.length; + while ( j-- ) { + handleObj = handlers[ j ]; + + if ( ( mappedTypes || origType === handleObj.origType ) && + ( !handler || handler.guid === handleObj.guid ) && + ( !tmp || tmp.test( handleObj.namespace ) ) && + ( !selector || selector === handleObj.selector || + selector === "**" && handleObj.selector ) ) { + handlers.splice( j, 1 ); + + if ( handleObj.selector ) { + handlers.delegateCount--; + } + if ( special.remove ) { + special.remove.call( elem, handleObj ); + } + } + } + + // Remove generic event handler if we removed something and no more handlers exist + // (avoids potential for endless recursion during removal of special event handlers) + if ( origCount && !handlers.length ) { + if ( !special.teardown || + special.teardown.call( elem, namespaces, elemData.handle ) === false ) { + + jQuery.removeEvent( elem, type, elemData.handle ); + } + + delete events[ type ]; + } + } + + // Remove data and the expando if it's no longer used + if ( jQuery.isEmptyObject( events ) ) { + dataPriv.remove( elem, "handle events" ); + } + }, + + dispatch: function( nativeEvent ) { + + // Make a writable jQuery.Event from the native event object + var event = jQuery.event.fix( nativeEvent ); + + var i, j, ret, matched, handleObj, handlerQueue, + args = new Array( arguments.length ), + handlers = ( dataPriv.get( this, "events" ) || {} )[ event.type ] || [], + special = jQuery.event.special[ event.type ] || {}; + + // Use the fix-ed jQuery.Event rather than the (read-only) native event + args[ 0 ] = event; + + for ( i = 1; i < arguments.length; i++ ) { + args[ i ] = arguments[ i ]; + } + + event.delegateTarget = this; + + // Call the preDispatch hook for the mapped type, and let it bail if desired + if ( special.preDispatch && special.preDispatch.call( this, event ) === false ) { + return; + } + + // Determine handlers + handlerQueue = jQuery.event.handlers.call( this, event, handlers ); + + // Run delegates first; they may want to stop propagation beneath us + i = 0; + while ( ( matched = handlerQueue[ i++ ] ) && !event.isPropagationStopped() ) { + event.currentTarget = matched.elem; + + j = 0; + while ( ( handleObj = matched.handlers[ j++ ] ) && + !event.isImmediatePropagationStopped() ) { + + // If the event is namespaced, then each handler is only invoked if it is + // specially universal or its namespaces are a superset of the event's. + if ( !event.rnamespace || handleObj.namespace === false || + event.rnamespace.test( handleObj.namespace ) ) { + + event.handleObj = handleObj; + event.data = handleObj.data; + + ret = ( ( jQuery.event.special[ handleObj.origType ] || {} ).handle || + handleObj.handler ).apply( matched.elem, args ); + + if ( ret !== undefined ) { + if ( ( event.result = ret ) === false ) { + event.preventDefault(); + event.stopPropagation(); + } + } + } + } + } + + // Call the postDispatch hook for the mapped type + if ( special.postDispatch ) { + special.postDispatch.call( this, event ); + } + + return event.result; + }, + + handlers: function( event, handlers ) { + var i, handleObj, sel, matchedHandlers, matchedSelectors, + handlerQueue = [], + delegateCount = handlers.delegateCount, + cur = event.target; + + // Find delegate handlers + if ( delegateCount && + + // Support: IE <=9 + // Black-hole SVG instance trees (trac-13180) + cur.nodeType && + + // Support: Firefox <=42 + // Suppress spec-violating clicks indicating a non-primary pointer button (trac-3861) + // https://www.w3.org/TR/DOM-Level-3-Events/#event-type-click + // Support: IE 11 only + // ...but not arrow key "clicks" of radio inputs, which can have `button` -1 (gh-2343) + !( event.type === "click" && event.button >= 1 ) ) { + + for ( ; cur !== this; cur = cur.parentNode || this ) { + + // Don't check non-elements (#13208) + // Don't process clicks on disabled elements (#6911, #8165, #11382, #11764) + if ( cur.nodeType === 1 && !( event.type === "click" && cur.disabled === true ) ) { + matchedHandlers = []; + matchedSelectors = {}; + for ( i = 0; i < delegateCount; i++ ) { + handleObj = handlers[ i ]; + + // Don't conflict with Object.prototype properties (#13203) + sel = handleObj.selector + " "; + + if ( matchedSelectors[ sel ] === undefined ) { + matchedSelectors[ sel ] = handleObj.needsContext ? + jQuery( sel, this ).index( cur ) > -1 : + jQuery.find( sel, this, null, [ cur ] ).length; + } + if ( matchedSelectors[ sel ] ) { + matchedHandlers.push( handleObj ); + } + } + if ( matchedHandlers.length ) { + handlerQueue.push( { elem: cur, handlers: matchedHandlers } ); + } + } + } + } + + // Add the remaining (directly-bound) handlers + cur = this; + if ( delegateCount < handlers.length ) { + handlerQueue.push( { elem: cur, handlers: handlers.slice( delegateCount ) } ); + } + + return handlerQueue; + }, + + addProp: function( name, hook ) { + Object.defineProperty( jQuery.Event.prototype, name, { + enumerable: true, + configurable: true, + + get: isFunction( hook ) ? + function() { + if ( this.originalEvent ) { + return hook( this.originalEvent ); + } + } : + function() { + if ( this.originalEvent ) { + return this.originalEvent[ name ]; + } + }, + + set: function( value ) { + Object.defineProperty( this, name, { + enumerable: true, + configurable: true, + writable: true, + value: value + } ); + } + } ); + }, + + fix: function( originalEvent ) { + return originalEvent[ jQuery.expando ] ? + originalEvent : + new jQuery.Event( originalEvent ); + }, + + special: { + load: { + + // Prevent triggered image.load events from bubbling to window.load + noBubble: true + }, + click: { + + // Utilize native event to ensure correct state for checkable inputs + setup: function( data ) { + + // For mutual compressibility with _default, replace `this` access with a local var. + // `|| data` is dead code meant only to preserve the variable through minification. + var el = this || data; + + // Claim the first handler + if ( rcheckableType.test( el.type ) && + el.click && nodeName( el, "input" ) ) { + + // dataPriv.set( el, "click", ... ) + leverageNative( el, "click", returnTrue ); + } + + // Return false to allow normal processing in the caller + return false; + }, + trigger: function( data ) { + + // For mutual compressibility with _default, replace `this` access with a local var. + // `|| data` is dead code meant only to preserve the variable through minification. + var el = this || data; + + // Force setup before triggering a click + if ( rcheckableType.test( el.type ) && + el.click && nodeName( el, "input" ) ) { + + leverageNative( el, "click" ); + } + + // Return non-false to allow normal event-path propagation + return true; + }, + + // For cross-browser consistency, suppress native .click() on links + // Also prevent it if we're currently inside a leveraged native-event stack + _default: function( event ) { + var target = event.target; + return rcheckableType.test( target.type ) && + target.click && nodeName( target, "input" ) && + dataPriv.get( target, "click" ) || + nodeName( target, "a" ); + } + }, + + beforeunload: { + postDispatch: function( event ) { + + // Support: Firefox 20+ + // Firefox doesn't alert if the returnValue field is not set. + if ( event.result !== undefined && event.originalEvent ) { + event.originalEvent.returnValue = event.result; + } + } + } + } +}; + +// Ensure the presence of an event listener that handles manually-triggered +// synthetic events by interrupting progress until reinvoked in response to +// *native* events that it fires directly, ensuring that state changes have +// already occurred before other listeners are invoked. +function leverageNative( el, type, expectSync ) { + + // Missing expectSync indicates a trigger call, which must force setup through jQuery.event.add + if ( !expectSync ) { + if ( dataPriv.get( el, type ) === undefined ) { + jQuery.event.add( el, type, returnTrue ); + } + return; + } + + // Register the controller as a special universal handler for all event namespaces + dataPriv.set( el, type, false ); + jQuery.event.add( el, type, { + namespace: false, + handler: function( event ) { + var notAsync, result, + saved = dataPriv.get( this, type ); + + if ( ( event.isTrigger & 1 ) && this[ type ] ) { + + // Interrupt processing of the outer synthetic .trigger()ed event + // Saved data should be false in such cases, but might be a leftover capture object + // from an async native handler (gh-4350) + if ( !saved.length ) { + + // Store arguments for use when handling the inner native event + // There will always be at least one argument (an event object), so this array + // will not be confused with a leftover capture object. + saved = slice.call( arguments ); + dataPriv.set( this, type, saved ); + + // Trigger the native event and capture its result + // Support: IE <=9 - 11+ + // focus() and blur() are asynchronous + notAsync = expectSync( this, type ); + this[ type ](); + result = dataPriv.get( this, type ); + if ( saved !== result || notAsync ) { + dataPriv.set( this, type, false ); + } else { + result = {}; + } + if ( saved !== result ) { + + // Cancel the outer synthetic event + event.stopImmediatePropagation(); + event.preventDefault(); + return result.value; + } + + // If this is an inner synthetic event for an event with a bubbling surrogate + // (focus or blur), assume that the surrogate already propagated from triggering the + // native event and prevent that from happening again here. + // This technically gets the ordering wrong w.r.t. to `.trigger()` (in which the + // bubbling surrogate propagates *after* the non-bubbling base), but that seems + // less bad than duplication. + } else if ( ( jQuery.event.special[ type ] || {} ).delegateType ) { + event.stopPropagation(); + } + + // If this is a native event triggered above, everything is now in order + // Fire an inner synthetic event with the original arguments + } else if ( saved.length ) { + + // ...and capture the result + dataPriv.set( this, type, { + value: jQuery.event.trigger( + + // Support: IE <=9 - 11+ + // Extend with the prototype to reset the above stopImmediatePropagation() + jQuery.extend( saved[ 0 ], jQuery.Event.prototype ), + saved.slice( 1 ), + this + ) + } ); + + // Abort handling of the native event + event.stopImmediatePropagation(); + } + } + } ); +} + +jQuery.removeEvent = function( elem, type, handle ) { + + // This "if" is needed for plain objects + if ( elem.removeEventListener ) { + elem.removeEventListener( type, handle ); + } +}; + +jQuery.Event = function( src, props ) { + + // Allow instantiation without the 'new' keyword + if ( !( this instanceof jQuery.Event ) ) { + return new jQuery.Event( src, props ); + } + + // Event object + if ( src && src.type ) { + this.originalEvent = src; + this.type = src.type; + + // Events bubbling up the document may have been marked as prevented + // by a handler lower down the tree; reflect the correct value. + this.isDefaultPrevented = src.defaultPrevented || + src.defaultPrevented === undefined && + + // Support: Android <=2.3 only + src.returnValue === false ? + returnTrue : + returnFalse; + + // Create target properties + // Support: Safari <=6 - 7 only + // Target should not be a text node (#504, #13143) + this.target = ( src.target && src.target.nodeType === 3 ) ? + src.target.parentNode : + src.target; + + this.currentTarget = src.currentTarget; + this.relatedTarget = src.relatedTarget; + + // Event type + } else { + this.type = src; + } + + // Put explicitly provided properties onto the event object + if ( props ) { + jQuery.extend( this, props ); + } + + // Create a timestamp if incoming event doesn't have one + this.timeStamp = src && src.timeStamp || Date.now(); + + // Mark it as fixed + this[ jQuery.expando ] = true; +}; + +// jQuery.Event is based on DOM3 Events as specified by the ECMAScript Language Binding +// https://www.w3.org/TR/2003/WD-DOM-Level-3-Events-20030331/ecma-script-binding.html +jQuery.Event.prototype = { + constructor: jQuery.Event, + isDefaultPrevented: returnFalse, + isPropagationStopped: returnFalse, + isImmediatePropagationStopped: returnFalse, + isSimulated: false, + + preventDefault: function() { + var e = this.originalEvent; + + this.isDefaultPrevented = returnTrue; + + if ( e && !this.isSimulated ) { + e.preventDefault(); + } + }, + stopPropagation: function() { + var e = this.originalEvent; + + this.isPropagationStopped = returnTrue; + + if ( e && !this.isSimulated ) { + e.stopPropagation(); + } + }, + stopImmediatePropagation: function() { + var e = this.originalEvent; + + this.isImmediatePropagationStopped = returnTrue; + + if ( e && !this.isSimulated ) { + e.stopImmediatePropagation(); + } + + this.stopPropagation(); + } +}; + +// Includes all common event props including KeyEvent and MouseEvent specific props +jQuery.each( { + altKey: true, + bubbles: true, + cancelable: true, + changedTouches: true, + ctrlKey: true, + detail: true, + eventPhase: true, + metaKey: true, + pageX: true, + pageY: true, + shiftKey: true, + view: true, + "char": true, + code: true, + charCode: true, + key: true, + keyCode: true, + button: true, + buttons: true, + clientX: true, + clientY: true, + offsetX: true, + offsetY: true, + pointerId: true, + pointerType: true, + screenX: true, + screenY: true, + targetTouches: true, + toElement: true, + touches: true, + + which: function( event ) { + var button = event.button; + + // Add which for key events + if ( event.which == null && rkeyEvent.test( event.type ) ) { + return event.charCode != null ? event.charCode : event.keyCode; + } + + // Add which for click: 1 === left; 2 === middle; 3 === right + if ( !event.which && button !== undefined && rmouseEvent.test( event.type ) ) { + if ( button & 1 ) { + return 1; + } + + if ( button & 2 ) { + return 3; + } + + if ( button & 4 ) { + return 2; + } + + return 0; + } + + return event.which; + } +}, jQuery.event.addProp ); + +jQuery.each( { focus: "focusin", blur: "focusout" }, function( type, delegateType ) { + jQuery.event.special[ type ] = { + + // Utilize native event if possible so blur/focus sequence is correct + setup: function() { + + // Claim the first handler + // dataPriv.set( this, "focus", ... ) + // dataPriv.set( this, "blur", ... ) + leverageNative( this, type, expectSync ); + + // Return false to allow normal processing in the caller + return false; + }, + trigger: function() { + + // Force setup before trigger + leverageNative( this, type ); + + // Return non-false to allow normal event-path propagation + return true; + }, + + delegateType: delegateType + }; +} ); + +// Create mouseenter/leave events using mouseover/out and event-time checks +// so that event delegation works in jQuery. +// Do the same for pointerenter/pointerleave and pointerover/pointerout +// +// Support: Safari 7 only +// Safari sends mouseenter too often; see: +// https://bugs.chromium.org/p/chromium/issues/detail?id=470258 +// for the description of the bug (it existed in older Chrome versions as well). +jQuery.each( { + mouseenter: "mouseover", + mouseleave: "mouseout", + pointerenter: "pointerover", + pointerleave: "pointerout" +}, function( orig, fix ) { + jQuery.event.special[ orig ] = { + delegateType: fix, + bindType: fix, + + handle: function( event ) { + var ret, + target = this, + related = event.relatedTarget, + handleObj = event.handleObj; + + // For mouseenter/leave call the handler if related is outside the target. + // NB: No relatedTarget if the mouse left/entered the browser window + if ( !related || ( related !== target && !jQuery.contains( target, related ) ) ) { + event.type = handleObj.origType; + ret = handleObj.handler.apply( this, arguments ); + event.type = fix; + } + return ret; + } + }; +} ); + +jQuery.fn.extend( { + + on: function( types, selector, data, fn ) { + return on( this, types, selector, data, fn ); + }, + one: function( types, selector, data, fn ) { + return on( this, types, selector, data, fn, 1 ); + }, + off: function( types, selector, fn ) { + var handleObj, type; + if ( types && types.preventDefault && types.handleObj ) { + + // ( event ) dispatched jQuery.Event + handleObj = types.handleObj; + jQuery( types.delegateTarget ).off( + handleObj.namespace ? + handleObj.origType + "." + handleObj.namespace : + handleObj.origType, + handleObj.selector, + handleObj.handler + ); + return this; + } + if ( typeof types === "object" ) { + + // ( types-object [, selector] ) + for ( type in types ) { + this.off( type, selector, types[ type ] ); + } + return this; + } + if ( selector === false || typeof selector === "function" ) { + + // ( types [, fn] ) + fn = selector; + selector = undefined; + } + if ( fn === false ) { + fn = returnFalse; + } + return this.each( function() { + jQuery.event.remove( this, types, fn, selector ); + } ); + } +} ); + + +var + + /* eslint-disable max-len */ + + // See https://github.com/eslint/eslint/issues/3229 + rxhtmlTag = /<(?!area|br|col|embed|hr|img|input|link|meta|param)(([a-z][^\/\0>\x20\t\r\n\f]*)[^>]*)\/>/gi, + + /* eslint-enable */ + + // Support: IE <=10 - 11, Edge 12 - 13 only + // In IE/Edge using regex groups here causes severe slowdowns. + // See https://connect.microsoft.com/IE/feedback/details/1736512/ + rnoInnerhtml = /\s*$/g; + +// Prefer a tbody over its parent table for containing new rows +function manipulationTarget( elem, content ) { + if ( nodeName( elem, "table" ) && + nodeName( content.nodeType !== 11 ? content : content.firstChild, "tr" ) ) { + + return jQuery( elem ).children( "tbody" )[ 0 ] || elem; + } + + return elem; +} + +// Replace/restore the type attribute of script elements for safe DOM manipulation +function disableScript( elem ) { + elem.type = ( elem.getAttribute( "type" ) !== null ) + "/" + elem.type; + return elem; +} +function restoreScript( elem ) { + if ( ( elem.type || "" ).slice( 0, 5 ) === "true/" ) { + elem.type = elem.type.slice( 5 ); + } else { + elem.removeAttribute( "type" ); + } + + return elem; +} + +function cloneCopyEvent( src, dest ) { + var i, l, type, pdataOld, pdataCur, udataOld, udataCur, events; + + if ( dest.nodeType !== 1 ) { + return; + } + + // 1. Copy private data: events, handlers, etc. + if ( dataPriv.hasData( src ) ) { + pdataOld = dataPriv.access( src ); + pdataCur = dataPriv.set( dest, pdataOld ); + events = pdataOld.events; + + if ( events ) { + delete pdataCur.handle; + pdataCur.events = {}; + + for ( type in events ) { + for ( i = 0, l = events[ type ].length; i < l; i++ ) { + jQuery.event.add( dest, type, events[ type ][ i ] ); + } + } + } + } + + // 2. Copy user data + if ( dataUser.hasData( src ) ) { + udataOld = dataUser.access( src ); + udataCur = jQuery.extend( {}, udataOld ); + + dataUser.set( dest, udataCur ); + } +} + +// Fix IE bugs, see support tests +function fixInput( src, dest ) { + var nodeName = dest.nodeName.toLowerCase(); + + // Fails to persist the checked state of a cloned checkbox or radio button. + if ( nodeName === "input" && rcheckableType.test( src.type ) ) { + dest.checked = src.checked; + + // Fails to return the selected option to the default selected state when cloning options + } else if ( nodeName === "input" || nodeName === "textarea" ) { + dest.defaultValue = src.defaultValue; + } +} + +function domManip( collection, args, callback, ignored ) { + + // Flatten any nested arrays + args = concat.apply( [], args ); + + var fragment, first, scripts, hasScripts, node, doc, + i = 0, + l = collection.length, + iNoClone = l - 1, + value = args[ 0 ], + valueIsFunction = isFunction( value ); + + // We can't cloneNode fragments that contain checked, in WebKit + if ( valueIsFunction || + ( l > 1 && typeof value === "string" && + !support.checkClone && rchecked.test( value ) ) ) { + return collection.each( function( index ) { + var self = collection.eq( index ); + if ( valueIsFunction ) { + args[ 0 ] = value.call( this, index, self.html() ); + } + domManip( self, args, callback, ignored ); + } ); + } + + if ( l ) { + fragment = buildFragment( args, collection[ 0 ].ownerDocument, false, collection, ignored ); + first = fragment.firstChild; + + if ( fragment.childNodes.length === 1 ) { + fragment = first; + } + + // Require either new content or an interest in ignored elements to invoke the callback + if ( first || ignored ) { + scripts = jQuery.map( getAll( fragment, "script" ), disableScript ); + hasScripts = scripts.length; + + // Use the original fragment for the last item + // instead of the first because it can end up + // being emptied incorrectly in certain situations (#8070). + for ( ; i < l; i++ ) { + node = fragment; + + if ( i !== iNoClone ) { + node = jQuery.clone( node, true, true ); + + // Keep references to cloned scripts for later restoration + if ( hasScripts ) { + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + jQuery.merge( scripts, getAll( node, "script" ) ); + } + } + + callback.call( collection[ i ], node, i ); + } + + if ( hasScripts ) { + doc = scripts[ scripts.length - 1 ].ownerDocument; + + // Reenable scripts + jQuery.map( scripts, restoreScript ); + + // Evaluate executable scripts on first document insertion + for ( i = 0; i < hasScripts; i++ ) { + node = scripts[ i ]; + if ( rscriptType.test( node.type || "" ) && + !dataPriv.access( node, "globalEval" ) && + jQuery.contains( doc, node ) ) { + + if ( node.src && ( node.type || "" ).toLowerCase() !== "module" ) { + + // Optional AJAX dependency, but won't run scripts if not present + if ( jQuery._evalUrl && !node.noModule ) { + jQuery._evalUrl( node.src, { + nonce: node.nonce || node.getAttribute( "nonce" ) + } ); + } + } else { + DOMEval( node.textContent.replace( rcleanScript, "" ), node, doc ); + } + } + } + } + } + } + + return collection; +} + +function remove( elem, selector, keepData ) { + var node, + nodes = selector ? jQuery.filter( selector, elem ) : elem, + i = 0; + + for ( ; ( node = nodes[ i ] ) != null; i++ ) { + if ( !keepData && node.nodeType === 1 ) { + jQuery.cleanData( getAll( node ) ); + } + + if ( node.parentNode ) { + if ( keepData && isAttached( node ) ) { + setGlobalEval( getAll( node, "script" ) ); + } + node.parentNode.removeChild( node ); + } + } + + return elem; +} + +jQuery.extend( { + htmlPrefilter: function( html ) { + return html.replace( rxhtmlTag, "<$1>" ); + }, + + clone: function( elem, dataAndEvents, deepDataAndEvents ) { + var i, l, srcElements, destElements, + clone = elem.cloneNode( true ), + inPage = isAttached( elem ); + + // Fix IE cloning issues + if ( !support.noCloneChecked && ( elem.nodeType === 1 || elem.nodeType === 11 ) && + !jQuery.isXMLDoc( elem ) ) { + + // We eschew Sizzle here for performance reasons: https://jsperf.com/getall-vs-sizzle/2 + destElements = getAll( clone ); + srcElements = getAll( elem ); + + for ( i = 0, l = srcElements.length; i < l; i++ ) { + fixInput( srcElements[ i ], destElements[ i ] ); + } + } + + // Copy the events from the original to the clone + if ( dataAndEvents ) { + if ( deepDataAndEvents ) { + srcElements = srcElements || getAll( elem ); + destElements = destElements || getAll( clone ); + + for ( i = 0, l = srcElements.length; i < l; i++ ) { + cloneCopyEvent( srcElements[ i ], destElements[ i ] ); + } + } else { + cloneCopyEvent( elem, clone ); + } + } + + // Preserve script evaluation history + destElements = getAll( clone, "script" ); + if ( destElements.length > 0 ) { + setGlobalEval( destElements, !inPage && getAll( elem, "script" ) ); + } + + // Return the cloned set + return clone; + }, + + cleanData: function( elems ) { + var data, elem, type, + special = jQuery.event.special, + i = 0; + + for ( ; ( elem = elems[ i ] ) !== undefined; i++ ) { + if ( acceptData( elem ) ) { + if ( ( data = elem[ dataPriv.expando ] ) ) { + if ( data.events ) { + for ( type in data.events ) { + if ( special[ type ] ) { + jQuery.event.remove( elem, type ); + + // This is a shortcut to avoid jQuery.event.remove's overhead + } else { + jQuery.removeEvent( elem, type, data.handle ); + } + } + } + + // Support: Chrome <=35 - 45+ + // Assign undefined instead of using delete, see Data#remove + elem[ dataPriv.expando ] = undefined; + } + if ( elem[ dataUser.expando ] ) { + + // Support: Chrome <=35 - 45+ + // Assign undefined instead of using delete, see Data#remove + elem[ dataUser.expando ] = undefined; + } + } + } + } +} ); + +jQuery.fn.extend( { + detach: function( selector ) { + return remove( this, selector, true ); + }, + + remove: function( selector ) { + return remove( this, selector ); + }, + + text: function( value ) { + return access( this, function( value ) { + return value === undefined ? + jQuery.text( this ) : + this.empty().each( function() { + if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) { + this.textContent = value; + } + } ); + }, null, value, arguments.length ); + }, + + append: function() { + return domManip( this, arguments, function( elem ) { + if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) { + var target = manipulationTarget( this, elem ); + target.appendChild( elem ); + } + } ); + }, + + prepend: function() { + return domManip( this, arguments, function( elem ) { + if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) { + var target = manipulationTarget( this, elem ); + target.insertBefore( elem, target.firstChild ); + } + } ); + }, + + before: function() { + return domManip( this, arguments, function( elem ) { + if ( this.parentNode ) { + this.parentNode.insertBefore( elem, this ); + } + } ); + }, + + after: function() { + return domManip( this, arguments, function( elem ) { + if ( this.parentNode ) { + this.parentNode.insertBefore( elem, this.nextSibling ); + } + } ); + }, + + empty: function() { + var elem, + i = 0; + + for ( ; ( elem = this[ i ] ) != null; i++ ) { + if ( elem.nodeType === 1 ) { + + // Prevent memory leaks + jQuery.cleanData( getAll( elem, false ) ); + + // Remove any remaining nodes + elem.textContent = ""; + } + } + + return this; + }, + + clone: function( dataAndEvents, deepDataAndEvents ) { + dataAndEvents = dataAndEvents == null ? false : dataAndEvents; + deepDataAndEvents = deepDataAndEvents == null ? dataAndEvents : deepDataAndEvents; + + return this.map( function() { + return jQuery.clone( this, dataAndEvents, deepDataAndEvents ); + } ); + }, + + html: function( value ) { + return access( this, function( value ) { + var elem = this[ 0 ] || {}, + i = 0, + l = this.length; + + if ( value === undefined && elem.nodeType === 1 ) { + return elem.innerHTML; + } + + // See if we can take a shortcut and just use innerHTML + if ( typeof value === "string" && !rnoInnerhtml.test( value ) && + !wrapMap[ ( rtagName.exec( value ) || [ "", "" ] )[ 1 ].toLowerCase() ] ) { + + value = jQuery.htmlPrefilter( value ); + + try { + for ( ; i < l; i++ ) { + elem = this[ i ] || {}; + + // Remove element nodes and prevent memory leaks + if ( elem.nodeType === 1 ) { + jQuery.cleanData( getAll( elem, false ) ); + elem.innerHTML = value; + } + } + + elem = 0; + + // If using innerHTML throws an exception, use the fallback method + } catch ( e ) {} + } + + if ( elem ) { + this.empty().append( value ); + } + }, null, value, arguments.length ); + }, + + replaceWith: function() { + var ignored = []; + + // Make the changes, replacing each non-ignored context element with the new content + return domManip( this, arguments, function( elem ) { + var parent = this.parentNode; + + if ( jQuery.inArray( this, ignored ) < 0 ) { + jQuery.cleanData( getAll( this ) ); + if ( parent ) { + parent.replaceChild( elem, this ); + } + } + + // Force callback invocation + }, ignored ); + } +} ); + +jQuery.each( { + appendTo: "append", + prependTo: "prepend", + insertBefore: "before", + insertAfter: "after", + replaceAll: "replaceWith" +}, function( name, original ) { + jQuery.fn[ name ] = function( selector ) { + var elems, + ret = [], + insert = jQuery( selector ), + last = insert.length - 1, + i = 0; + + for ( ; i <= last; i++ ) { + elems = i === last ? this : this.clone( true ); + jQuery( insert[ i ] )[ original ]( elems ); + + // Support: Android <=4.0 only, PhantomJS 1 only + // .get() because push.apply(_, arraylike) throws on ancient WebKit + push.apply( ret, elems.get() ); + } + + return this.pushStack( ret ); + }; +} ); +var rnumnonpx = new RegExp( "^(" + pnum + ")(?!px)[a-z%]+$", "i" ); + +var getStyles = function( elem ) { + + // Support: IE <=11 only, Firefox <=30 (#15098, #14150) + // IE throws on elements created in popups + // FF meanwhile throws on frame elements through "defaultView.getComputedStyle" + var view = elem.ownerDocument.defaultView; + + if ( !view || !view.opener ) { + view = window; + } + + return view.getComputedStyle( elem ); + }; + +var rboxStyle = new RegExp( cssExpand.join( "|" ), "i" ); + + + +( function() { + + // Executing both pixelPosition & boxSizingReliable tests require only one layout + // so they're executed at the same time to save the second computation. + function computeStyleTests() { + + // This is a singleton, we need to execute it only once + if ( !div ) { + return; + } + + container.style.cssText = "position:absolute;left:-11111px;width:60px;" + + "margin-top:1px;padding:0;border:0"; + div.style.cssText = + "position:relative;display:block;box-sizing:border-box;overflow:scroll;" + + "margin:auto;border:1px;padding:1px;" + + "width:60%;top:1%"; + documentElement.appendChild( container ).appendChild( div ); + + var divStyle = window.getComputedStyle( div ); + pixelPositionVal = divStyle.top !== "1%"; + + // Support: Android 4.0 - 4.3 only, Firefox <=3 - 44 + reliableMarginLeftVal = roundPixelMeasures( divStyle.marginLeft ) === 12; + + // Support: Android 4.0 - 4.3 only, Safari <=9.1 - 10.1, iOS <=7.0 - 9.3 + // Some styles come back with percentage values, even though they shouldn't + div.style.right = "60%"; + pixelBoxStylesVal = roundPixelMeasures( divStyle.right ) === 36; + + // Support: IE 9 - 11 only + // Detect misreporting of content dimensions for box-sizing:border-box elements + boxSizingReliableVal = roundPixelMeasures( divStyle.width ) === 36; + + // Support: IE 9 only + // Detect overflow:scroll screwiness (gh-3699) + // Support: Chrome <=64 + // Don't get tricked when zoom affects offsetWidth (gh-4029) + div.style.position = "absolute"; + scrollboxSizeVal = roundPixelMeasures( div.offsetWidth / 3 ) === 12; + + documentElement.removeChild( container ); + + // Nullify the div so it wouldn't be stored in the memory and + // it will also be a sign that checks already performed + div = null; + } + + function roundPixelMeasures( measure ) { + return Math.round( parseFloat( measure ) ); + } + + var pixelPositionVal, boxSizingReliableVal, scrollboxSizeVal, pixelBoxStylesVal, + reliableMarginLeftVal, + container = document.createElement( "div" ), + div = document.createElement( "div" ); + + // Finish early in limited (non-browser) environments + if ( !div.style ) { + return; + } + + // Support: IE <=9 - 11 only + // Style of cloned element affects source element cloned (#8908) + div.style.backgroundClip = "content-box"; + div.cloneNode( true ).style.backgroundClip = ""; + support.clearCloneStyle = div.style.backgroundClip === "content-box"; + + jQuery.extend( support, { + boxSizingReliable: function() { + computeStyleTests(); + return boxSizingReliableVal; + }, + pixelBoxStyles: function() { + computeStyleTests(); + return pixelBoxStylesVal; + }, + pixelPosition: function() { + computeStyleTests(); + return pixelPositionVal; + }, + reliableMarginLeft: function() { + computeStyleTests(); + return reliableMarginLeftVal; + }, + scrollboxSize: function() { + computeStyleTests(); + return scrollboxSizeVal; + } + } ); +} )(); + + +function curCSS( elem, name, computed ) { + var width, minWidth, maxWidth, ret, + + // Support: Firefox 51+ + // Retrieving style before computed somehow + // fixes an issue with getting wrong values + // on detached elements + style = elem.style; + + computed = computed || getStyles( elem ); + + // getPropertyValue is needed for: + // .css('filter') (IE 9 only, #12537) + // .css('--customProperty) (#3144) + if ( computed ) { + ret = computed.getPropertyValue( name ) || computed[ name ]; + + if ( ret === "" && !isAttached( elem ) ) { + ret = jQuery.style( elem, name ); + } + + // A tribute to the "awesome hack by Dean Edwards" + // Android Browser returns percentage for some values, + // but width seems to be reliably pixels. + // This is against the CSSOM draft spec: + // https://drafts.csswg.org/cssom/#resolved-values + if ( !support.pixelBoxStyles() && rnumnonpx.test( ret ) && rboxStyle.test( name ) ) { + + // Remember the original values + width = style.width; + minWidth = style.minWidth; + maxWidth = style.maxWidth; + + // Put in the new values to get a computed value out + style.minWidth = style.maxWidth = style.width = ret; + ret = computed.width; + + // Revert the changed values + style.width = width; + style.minWidth = minWidth; + style.maxWidth = maxWidth; + } + } + + return ret !== undefined ? + + // Support: IE <=9 - 11 only + // IE returns zIndex value as an integer. + ret + "" : + ret; +} + + +function addGetHookIf( conditionFn, hookFn ) { + + // Define the hook, we'll check on the first run if it's really needed. + return { + get: function() { + if ( conditionFn() ) { + + // Hook not needed (or it's not possible to use it due + // to missing dependency), remove it. + delete this.get; + return; + } + + // Hook needed; redefine it so that the support test is not executed again. + return ( this.get = hookFn ).apply( this, arguments ); + } + }; +} + + +var cssPrefixes = [ "Webkit", "Moz", "ms" ], + emptyStyle = document.createElement( "div" ).style, + vendorProps = {}; + +// Return a vendor-prefixed property or undefined +function vendorPropName( name ) { + + // Check for vendor prefixed names + var capName = name[ 0 ].toUpperCase() + name.slice( 1 ), + i = cssPrefixes.length; + + while ( i-- ) { + name = cssPrefixes[ i ] + capName; + if ( name in emptyStyle ) { + return name; + } + } +} + +// Return a potentially-mapped jQuery.cssProps or vendor prefixed property +function finalPropName( name ) { + var final = jQuery.cssProps[ name ] || vendorProps[ name ]; + + if ( final ) { + return final; + } + if ( name in emptyStyle ) { + return name; + } + return vendorProps[ name ] = vendorPropName( name ) || name; +} + + +var + + // Swappable if display is none or starts with table + // except "table", "table-cell", or "table-caption" + // See here for display values: https://developer.mozilla.org/en-US/docs/CSS/display + rdisplayswap = /^(none|table(?!-c[ea]).+)/, + rcustomProp = /^--/, + cssShow = { position: "absolute", visibility: "hidden", display: "block" }, + cssNormalTransform = { + letterSpacing: "0", + fontWeight: "400" + }; + +function setPositiveNumber( elem, value, subtract ) { + + // Any relative (+/-) values have already been + // normalized at this point + var matches = rcssNum.exec( value ); + return matches ? + + // Guard against undefined "subtract", e.g., when used as in cssHooks + Math.max( 0, matches[ 2 ] - ( subtract || 0 ) ) + ( matches[ 3 ] || "px" ) : + value; +} + +function boxModelAdjustment( elem, dimension, box, isBorderBox, styles, computedVal ) { + var i = dimension === "width" ? 1 : 0, + extra = 0, + delta = 0; + + // Adjustment may not be necessary + if ( box === ( isBorderBox ? "border" : "content" ) ) { + return 0; + } + + for ( ; i < 4; i += 2 ) { + + // Both box models exclude margin + if ( box === "margin" ) { + delta += jQuery.css( elem, box + cssExpand[ i ], true, styles ); + } + + // If we get here with a content-box, we're seeking "padding" or "border" or "margin" + if ( !isBorderBox ) { + + // Add padding + delta += jQuery.css( elem, "padding" + cssExpand[ i ], true, styles ); + + // For "border" or "margin", add border + if ( box !== "padding" ) { + delta += jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles ); + + // But still keep track of it otherwise + } else { + extra += jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles ); + } + + // If we get here with a border-box (content + padding + border), we're seeking "content" or + // "padding" or "margin" + } else { + + // For "content", subtract padding + if ( box === "content" ) { + delta -= jQuery.css( elem, "padding" + cssExpand[ i ], true, styles ); + } + + // For "content" or "padding", subtract border + if ( box !== "margin" ) { + delta -= jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles ); + } + } + } + + // Account for positive content-box scroll gutter when requested by providing computedVal + if ( !isBorderBox && computedVal >= 0 ) { + + // offsetWidth/offsetHeight is a rounded sum of content, padding, scroll gutter, and border + // Assuming integer scroll gutter, subtract the rest and round down + delta += Math.max( 0, Math.ceil( + elem[ "offset" + dimension[ 0 ].toUpperCase() + dimension.slice( 1 ) ] - + computedVal - + delta - + extra - + 0.5 + + // If offsetWidth/offsetHeight is unknown, then we can't determine content-box scroll gutter + // Use an explicit zero to avoid NaN (gh-3964) + ) ) || 0; + } + + return delta; +} + +function getWidthOrHeight( elem, dimension, extra ) { + + // Start with computed style + var styles = getStyles( elem ), + + // To avoid forcing a reflow, only fetch boxSizing if we need it (gh-4322). + // Fake content-box until we know it's needed to know the true value. + boxSizingNeeded = !support.boxSizingReliable() || extra, + isBorderBox = boxSizingNeeded && + jQuery.css( elem, "boxSizing", false, styles ) === "border-box", + valueIsBorderBox = isBorderBox, + + val = curCSS( elem, dimension, styles ), + offsetProp = "offset" + dimension[ 0 ].toUpperCase() + dimension.slice( 1 ); + + // Support: Firefox <=54 + // Return a confounding non-pixel value or feign ignorance, as appropriate. + if ( rnumnonpx.test( val ) ) { + if ( !extra ) { + return val; + } + val = "auto"; + } + + + // Fall back to offsetWidth/offsetHeight when value is "auto" + // This happens for inline elements with no explicit setting (gh-3571) + // Support: Android <=4.1 - 4.3 only + // Also use offsetWidth/offsetHeight for misreported inline dimensions (gh-3602) + // Support: IE 9-11 only + // Also use offsetWidth/offsetHeight for when box sizing is unreliable + // We use getClientRects() to check for hidden/disconnected. + // In those cases, the computed value can be trusted to be border-box + if ( ( !support.boxSizingReliable() && isBorderBox || + val === "auto" || + !parseFloat( val ) && jQuery.css( elem, "display", false, styles ) === "inline" ) && + elem.getClientRects().length ) { + + isBorderBox = jQuery.css( elem, "boxSizing", false, styles ) === "border-box"; + + // Where available, offsetWidth/offsetHeight approximate border box dimensions. + // Where not available (e.g., SVG), assume unreliable box-sizing and interpret the + // retrieved value as a content box dimension. + valueIsBorderBox = offsetProp in elem; + if ( valueIsBorderBox ) { + val = elem[ offsetProp ]; + } + } + + // Normalize "" and auto + val = parseFloat( val ) || 0; + + // Adjust for the element's box model + return ( val + + boxModelAdjustment( + elem, + dimension, + extra || ( isBorderBox ? "border" : "content" ), + valueIsBorderBox, + styles, + + // Provide the current computed size to request scroll gutter calculation (gh-3589) + val + ) + ) + "px"; +} + +jQuery.extend( { + + // Add in style property hooks for overriding the default + // behavior of getting and setting a style property + cssHooks: { + opacity: { + get: function( elem, computed ) { + if ( computed ) { + + // We should always get a number back from opacity + var ret = curCSS( elem, "opacity" ); + return ret === "" ? "1" : ret; + } + } + } + }, + + // Don't automatically add "px" to these possibly-unitless properties + cssNumber: { + "animationIterationCount": true, + "columnCount": true, + "fillOpacity": true, + "flexGrow": true, + "flexShrink": true, + "fontWeight": true, + "gridArea": true, + "gridColumn": true, + "gridColumnEnd": true, + "gridColumnStart": true, + "gridRow": true, + "gridRowEnd": true, + "gridRowStart": true, + "lineHeight": true, + "opacity": true, + "order": true, + "orphans": true, + "widows": true, + "zIndex": true, + "zoom": true + }, + + // Add in properties whose names you wish to fix before + // setting or getting the value + cssProps: {}, + + // Get and set the style property on a DOM Node + style: function( elem, name, value, extra ) { + + // Don't set styles on text and comment nodes + if ( !elem || elem.nodeType === 3 || elem.nodeType === 8 || !elem.style ) { + return; + } + + // Make sure that we're working with the right name + var ret, type, hooks, + origName = camelCase( name ), + isCustomProp = rcustomProp.test( name ), + style = elem.style; + + // Make sure that we're working with the right name. We don't + // want to query the value if it is a CSS custom property + // since they are user-defined. + if ( !isCustomProp ) { + name = finalPropName( origName ); + } + + // Gets hook for the prefixed version, then unprefixed version + hooks = jQuery.cssHooks[ name ] || jQuery.cssHooks[ origName ]; + + // Check if we're setting a value + if ( value !== undefined ) { + type = typeof value; + + // Convert "+=" or "-=" to relative numbers (#7345) + if ( type === "string" && ( ret = rcssNum.exec( value ) ) && ret[ 1 ] ) { + value = adjustCSS( elem, name, ret ); + + // Fixes bug #9237 + type = "number"; + } + + // Make sure that null and NaN values aren't set (#7116) + if ( value == null || value !== value ) { + return; + } + + // If a number was passed in, add the unit (except for certain CSS properties) + // The isCustomProp check can be removed in jQuery 4.0 when we only auto-append + // "px" to a few hardcoded values. + if ( type === "number" && !isCustomProp ) { + value += ret && ret[ 3 ] || ( jQuery.cssNumber[ origName ] ? "" : "px" ); + } + + // background-* props affect original clone's values + if ( !support.clearCloneStyle && value === "" && name.indexOf( "background" ) === 0 ) { + style[ name ] = "inherit"; + } + + // If a hook was provided, use that value, otherwise just set the specified value + if ( !hooks || !( "set" in hooks ) || + ( value = hooks.set( elem, value, extra ) ) !== undefined ) { + + if ( isCustomProp ) { + style.setProperty( name, value ); + } else { + style[ name ] = value; + } + } + + } else { + + // If a hook was provided get the non-computed value from there + if ( hooks && "get" in hooks && + ( ret = hooks.get( elem, false, extra ) ) !== undefined ) { + + return ret; + } + + // Otherwise just get the value from the style object + return style[ name ]; + } + }, + + css: function( elem, name, extra, styles ) { + var val, num, hooks, + origName = camelCase( name ), + isCustomProp = rcustomProp.test( name ); + + // Make sure that we're working with the right name. We don't + // want to modify the value if it is a CSS custom property + // since they are user-defined. + if ( !isCustomProp ) { + name = finalPropName( origName ); + } + + // Try prefixed name followed by the unprefixed name + hooks = jQuery.cssHooks[ name ] || jQuery.cssHooks[ origName ]; + + // If a hook was provided get the computed value from there + if ( hooks && "get" in hooks ) { + val = hooks.get( elem, true, extra ); + } + + // Otherwise, if a way to get the computed value exists, use that + if ( val === undefined ) { + val = curCSS( elem, name, styles ); + } + + // Convert "normal" to computed value + if ( val === "normal" && name in cssNormalTransform ) { + val = cssNormalTransform[ name ]; + } + + // Make numeric if forced or a qualifier was provided and val looks numeric + if ( extra === "" || extra ) { + num = parseFloat( val ); + return extra === true || isFinite( num ) ? num || 0 : val; + } + + return val; + } +} ); + +jQuery.each( [ "height", "width" ], function( i, dimension ) { + jQuery.cssHooks[ dimension ] = { + get: function( elem, computed, extra ) { + if ( computed ) { + + // Certain elements can have dimension info if we invisibly show them + // but it must have a current display style that would benefit + return rdisplayswap.test( jQuery.css( elem, "display" ) ) && + + // Support: Safari 8+ + // Table columns in Safari have non-zero offsetWidth & zero + // getBoundingClientRect().width unless display is changed. + // Support: IE <=11 only + // Running getBoundingClientRect on a disconnected node + // in IE throws an error. + ( !elem.getClientRects().length || !elem.getBoundingClientRect().width ) ? + swap( elem, cssShow, function() { + return getWidthOrHeight( elem, dimension, extra ); + } ) : + getWidthOrHeight( elem, dimension, extra ); + } + }, + + set: function( elem, value, extra ) { + var matches, + styles = getStyles( elem ), + + // Only read styles.position if the test has a chance to fail + // to avoid forcing a reflow. + scrollboxSizeBuggy = !support.scrollboxSize() && + styles.position === "absolute", + + // To avoid forcing a reflow, only fetch boxSizing if we need it (gh-3991) + boxSizingNeeded = scrollboxSizeBuggy || extra, + isBorderBox = boxSizingNeeded && + jQuery.css( elem, "boxSizing", false, styles ) === "border-box", + subtract = extra ? + boxModelAdjustment( + elem, + dimension, + extra, + isBorderBox, + styles + ) : + 0; + + // Account for unreliable border-box dimensions by comparing offset* to computed and + // faking a content-box to get border and padding (gh-3699) + if ( isBorderBox && scrollboxSizeBuggy ) { + subtract -= Math.ceil( + elem[ "offset" + dimension[ 0 ].toUpperCase() + dimension.slice( 1 ) ] - + parseFloat( styles[ dimension ] ) - + boxModelAdjustment( elem, dimension, "border", false, styles ) - + 0.5 + ); + } + + // Convert to pixels if value adjustment is needed + if ( subtract && ( matches = rcssNum.exec( value ) ) && + ( matches[ 3 ] || "px" ) !== "px" ) { + + elem.style[ dimension ] = value; + value = jQuery.css( elem, dimension ); + } + + return setPositiveNumber( elem, value, subtract ); + } + }; +} ); + +jQuery.cssHooks.marginLeft = addGetHookIf( support.reliableMarginLeft, + function( elem, computed ) { + if ( computed ) { + return ( parseFloat( curCSS( elem, "marginLeft" ) ) || + elem.getBoundingClientRect().left - + swap( elem, { marginLeft: 0 }, function() { + return elem.getBoundingClientRect().left; + } ) + ) + "px"; + } + } +); + +// These hooks are used by animate to expand properties +jQuery.each( { + margin: "", + padding: "", + border: "Width" +}, function( prefix, suffix ) { + jQuery.cssHooks[ prefix + suffix ] = { + expand: function( value ) { + var i = 0, + expanded = {}, + + // Assumes a single number if not a string + parts = typeof value === "string" ? value.split( " " ) : [ value ]; + + for ( ; i < 4; i++ ) { + expanded[ prefix + cssExpand[ i ] + suffix ] = + parts[ i ] || parts[ i - 2 ] || parts[ 0 ]; + } + + return expanded; + } + }; + + if ( prefix !== "margin" ) { + jQuery.cssHooks[ prefix + suffix ].set = setPositiveNumber; + } +} ); + +jQuery.fn.extend( { + css: function( name, value ) { + return access( this, function( elem, name, value ) { + var styles, len, + map = {}, + i = 0; + + if ( Array.isArray( name ) ) { + styles = getStyles( elem ); + len = name.length; + + for ( ; i < len; i++ ) { + map[ name[ i ] ] = jQuery.css( elem, name[ i ], false, styles ); + } + + return map; + } + + return value !== undefined ? + jQuery.style( elem, name, value ) : + jQuery.css( elem, name ); + }, name, value, arguments.length > 1 ); + } +} ); + + +function Tween( elem, options, prop, end, easing ) { + return new Tween.prototype.init( elem, options, prop, end, easing ); +} +jQuery.Tween = Tween; + +Tween.prototype = { + constructor: Tween, + init: function( elem, options, prop, end, easing, unit ) { + this.elem = elem; + this.prop = prop; + this.easing = easing || jQuery.easing._default; + this.options = options; + this.start = this.now = this.cur(); + this.end = end; + this.unit = unit || ( jQuery.cssNumber[ prop ] ? "" : "px" ); + }, + cur: function() { + var hooks = Tween.propHooks[ this.prop ]; + + return hooks && hooks.get ? + hooks.get( this ) : + Tween.propHooks._default.get( this ); + }, + run: function( percent ) { + var eased, + hooks = Tween.propHooks[ this.prop ]; + + if ( this.options.duration ) { + this.pos = eased = jQuery.easing[ this.easing ]( + percent, this.options.duration * percent, 0, 1, this.options.duration + ); + } else { + this.pos = eased = percent; + } + this.now = ( this.end - this.start ) * eased + this.start; + + if ( this.options.step ) { + this.options.step.call( this.elem, this.now, this ); + } + + if ( hooks && hooks.set ) { + hooks.set( this ); + } else { + Tween.propHooks._default.set( this ); + } + return this; + } +}; + +Tween.prototype.init.prototype = Tween.prototype; + +Tween.propHooks = { + _default: { + get: function( tween ) { + var result; + + // Use a property on the element directly when it is not a DOM element, + // or when there is no matching style property that exists. + if ( tween.elem.nodeType !== 1 || + tween.elem[ tween.prop ] != null && tween.elem.style[ tween.prop ] == null ) { + return tween.elem[ tween.prop ]; + } + + // Passing an empty string as a 3rd parameter to .css will automatically + // attempt a parseFloat and fallback to a string if the parse fails. + // Simple values such as "10px" are parsed to Float; + // complex values such as "rotate(1rad)" are returned as-is. + result = jQuery.css( tween.elem, tween.prop, "" ); + + // Empty strings, null, undefined and "auto" are converted to 0. + return !result || result === "auto" ? 0 : result; + }, + set: function( tween ) { + + // Use step hook for back compat. + // Use cssHook if its there. + // Use .style if available and use plain properties where available. + if ( jQuery.fx.step[ tween.prop ] ) { + jQuery.fx.step[ tween.prop ]( tween ); + } else if ( tween.elem.nodeType === 1 && ( + jQuery.cssHooks[ tween.prop ] || + tween.elem.style[ finalPropName( tween.prop ) ] != null ) ) { + jQuery.style( tween.elem, tween.prop, tween.now + tween.unit ); + } else { + tween.elem[ tween.prop ] = tween.now; + } + } + } +}; + +// Support: IE <=9 only +// Panic based approach to setting things on disconnected nodes +Tween.propHooks.scrollTop = Tween.propHooks.scrollLeft = { + set: function( tween ) { + if ( tween.elem.nodeType && tween.elem.parentNode ) { + tween.elem[ tween.prop ] = tween.now; + } + } +}; + +jQuery.easing = { + linear: function( p ) { + return p; + }, + swing: function( p ) { + return 0.5 - Math.cos( p * Math.PI ) / 2; + }, + _default: "swing" +}; + +jQuery.fx = Tween.prototype.init; + +// Back compat <1.8 extension point +jQuery.fx.step = {}; + + + + +var + fxNow, inProgress, + rfxtypes = /^(?:toggle|show|hide)$/, + rrun = /queueHooks$/; + +function schedule() { + if ( inProgress ) { + if ( document.hidden === false && window.requestAnimationFrame ) { + window.requestAnimationFrame( schedule ); + } else { + window.setTimeout( schedule, jQuery.fx.interval ); + } + + jQuery.fx.tick(); + } +} + +// Animations created synchronously will run synchronously +function createFxNow() { + window.setTimeout( function() { + fxNow = undefined; + } ); + return ( fxNow = Date.now() ); +} + +// Generate parameters to create a standard animation +function genFx( type, includeWidth ) { + var which, + i = 0, + attrs = { height: type }; + + // If we include width, step value is 1 to do all cssExpand values, + // otherwise step value is 2 to skip over Left and Right + includeWidth = includeWidth ? 1 : 0; + for ( ; i < 4; i += 2 - includeWidth ) { + which = cssExpand[ i ]; + attrs[ "margin" + which ] = attrs[ "padding" + which ] = type; + } + + if ( includeWidth ) { + attrs.opacity = attrs.width = type; + } + + return attrs; +} + +function createTween( value, prop, animation ) { + var tween, + collection = ( Animation.tweeners[ prop ] || [] ).concat( Animation.tweeners[ "*" ] ), + index = 0, + length = collection.length; + for ( ; index < length; index++ ) { + if ( ( tween = collection[ index ].call( animation, prop, value ) ) ) { + + // We're done with this property + return tween; + } + } +} + +function defaultPrefilter( elem, props, opts ) { + var prop, value, toggle, hooks, oldfire, propTween, restoreDisplay, display, + isBox = "width" in props || "height" in props, + anim = this, + orig = {}, + style = elem.style, + hidden = elem.nodeType && isHiddenWithinTree( elem ), + dataShow = dataPriv.get( elem, "fxshow" ); + + // Queue-skipping animations hijack the fx hooks + if ( !opts.queue ) { + hooks = jQuery._queueHooks( elem, "fx" ); + if ( hooks.unqueued == null ) { + hooks.unqueued = 0; + oldfire = hooks.empty.fire; + hooks.empty.fire = function() { + if ( !hooks.unqueued ) { + oldfire(); + } + }; + } + hooks.unqueued++; + + anim.always( function() { + + // Ensure the complete handler is called before this completes + anim.always( function() { + hooks.unqueued--; + if ( !jQuery.queue( elem, "fx" ).length ) { + hooks.empty.fire(); + } + } ); + } ); + } + + // Detect show/hide animations + for ( prop in props ) { + value = props[ prop ]; + if ( rfxtypes.test( value ) ) { + delete props[ prop ]; + toggle = toggle || value === "toggle"; + if ( value === ( hidden ? "hide" : "show" ) ) { + + // Pretend to be hidden if this is a "show" and + // there is still data from a stopped show/hide + if ( value === "show" && dataShow && dataShow[ prop ] !== undefined ) { + hidden = true; + + // Ignore all other no-op show/hide data + } else { + continue; + } + } + orig[ prop ] = dataShow && dataShow[ prop ] || jQuery.style( elem, prop ); + } + } + + // Bail out if this is a no-op like .hide().hide() + propTween = !jQuery.isEmptyObject( props ); + if ( !propTween && jQuery.isEmptyObject( orig ) ) { + return; + } + + // Restrict "overflow" and "display" styles during box animations + if ( isBox && elem.nodeType === 1 ) { + + // Support: IE <=9 - 11, Edge 12 - 15 + // Record all 3 overflow attributes because IE does not infer the shorthand + // from identically-valued overflowX and overflowY and Edge just mirrors + // the overflowX value there. + opts.overflow = [ style.overflow, style.overflowX, style.overflowY ]; + + // Identify a display type, preferring old show/hide data over the CSS cascade + restoreDisplay = dataShow && dataShow.display; + if ( restoreDisplay == null ) { + restoreDisplay = dataPriv.get( elem, "display" ); + } + display = jQuery.css( elem, "display" ); + if ( display === "none" ) { + if ( restoreDisplay ) { + display = restoreDisplay; + } else { + + // Get nonempty value(s) by temporarily forcing visibility + showHide( [ elem ], true ); + restoreDisplay = elem.style.display || restoreDisplay; + display = jQuery.css( elem, "display" ); + showHide( [ elem ] ); + } + } + + // Animate inline elements as inline-block + if ( display === "inline" || display === "inline-block" && restoreDisplay != null ) { + if ( jQuery.css( elem, "float" ) === "none" ) { + + // Restore the original display value at the end of pure show/hide animations + if ( !propTween ) { + anim.done( function() { + style.display = restoreDisplay; + } ); + if ( restoreDisplay == null ) { + display = style.display; + restoreDisplay = display === "none" ? "" : display; + } + } + style.display = "inline-block"; + } + } + } + + if ( opts.overflow ) { + style.overflow = "hidden"; + anim.always( function() { + style.overflow = opts.overflow[ 0 ]; + style.overflowX = opts.overflow[ 1 ]; + style.overflowY = opts.overflow[ 2 ]; + } ); + } + + // Implement show/hide animations + propTween = false; + for ( prop in orig ) { + + // General show/hide setup for this element animation + if ( !propTween ) { + if ( dataShow ) { + if ( "hidden" in dataShow ) { + hidden = dataShow.hidden; + } + } else { + dataShow = dataPriv.access( elem, "fxshow", { display: restoreDisplay } ); + } + + // Store hidden/visible for toggle so `.stop().toggle()` "reverses" + if ( toggle ) { + dataShow.hidden = !hidden; + } + + // Show elements before animating them + if ( hidden ) { + showHide( [ elem ], true ); + } + + /* eslint-disable no-loop-func */ + + anim.done( function() { + + /* eslint-enable no-loop-func */ + + // The final step of a "hide" animation is actually hiding the element + if ( !hidden ) { + showHide( [ elem ] ); + } + dataPriv.remove( elem, "fxshow" ); + for ( prop in orig ) { + jQuery.style( elem, prop, orig[ prop ] ); + } + } ); + } + + // Per-property setup + propTween = createTween( hidden ? dataShow[ prop ] : 0, prop, anim ); + if ( !( prop in dataShow ) ) { + dataShow[ prop ] = propTween.start; + if ( hidden ) { + propTween.end = propTween.start; + propTween.start = 0; + } + } + } +} + +function propFilter( props, specialEasing ) { + var index, name, easing, value, hooks; + + // camelCase, specialEasing and expand cssHook pass + for ( index in props ) { + name = camelCase( index ); + easing = specialEasing[ name ]; + value = props[ index ]; + if ( Array.isArray( value ) ) { + easing = value[ 1 ]; + value = props[ index ] = value[ 0 ]; + } + + if ( index !== name ) { + props[ name ] = value; + delete props[ index ]; + } + + hooks = jQuery.cssHooks[ name ]; + if ( hooks && "expand" in hooks ) { + value = hooks.expand( value ); + delete props[ name ]; + + // Not quite $.extend, this won't overwrite existing keys. + // Reusing 'index' because we have the correct "name" + for ( index in value ) { + if ( !( index in props ) ) { + props[ index ] = value[ index ]; + specialEasing[ index ] = easing; + } + } + } else { + specialEasing[ name ] = easing; + } + } +} + +function Animation( elem, properties, options ) { + var result, + stopped, + index = 0, + length = Animation.prefilters.length, + deferred = jQuery.Deferred().always( function() { + + // Don't match elem in the :animated selector + delete tick.elem; + } ), + tick = function() { + if ( stopped ) { + return false; + } + var currentTime = fxNow || createFxNow(), + remaining = Math.max( 0, animation.startTime + animation.duration - currentTime ), + + // Support: Android 2.3 only + // Archaic crash bug won't allow us to use `1 - ( 0.5 || 0 )` (#12497) + temp = remaining / animation.duration || 0, + percent = 1 - temp, + index = 0, + length = animation.tweens.length; + + for ( ; index < length; index++ ) { + animation.tweens[ index ].run( percent ); + } + + deferred.notifyWith( elem, [ animation, percent, remaining ] ); + + // If there's more to do, yield + if ( percent < 1 && length ) { + return remaining; + } + + // If this was an empty animation, synthesize a final progress notification + if ( !length ) { + deferred.notifyWith( elem, [ animation, 1, 0 ] ); + } + + // Resolve the animation and report its conclusion + deferred.resolveWith( elem, [ animation ] ); + return false; + }, + animation = deferred.promise( { + elem: elem, + props: jQuery.extend( {}, properties ), + opts: jQuery.extend( true, { + specialEasing: {}, + easing: jQuery.easing._default + }, options ), + originalProperties: properties, + originalOptions: options, + startTime: fxNow || createFxNow(), + duration: options.duration, + tweens: [], + createTween: function( prop, end ) { + var tween = jQuery.Tween( elem, animation.opts, prop, end, + animation.opts.specialEasing[ prop ] || animation.opts.easing ); + animation.tweens.push( tween ); + return tween; + }, + stop: function( gotoEnd ) { + var index = 0, + + // If we are going to the end, we want to run all the tweens + // otherwise we skip this part + length = gotoEnd ? animation.tweens.length : 0; + if ( stopped ) { + return this; + } + stopped = true; + for ( ; index < length; index++ ) { + animation.tweens[ index ].run( 1 ); + } + + // Resolve when we played the last frame; otherwise, reject + if ( gotoEnd ) { + deferred.notifyWith( elem, [ animation, 1, 0 ] ); + deferred.resolveWith( elem, [ animation, gotoEnd ] ); + } else { + deferred.rejectWith( elem, [ animation, gotoEnd ] ); + } + return this; + } + } ), + props = animation.props; + + propFilter( props, animation.opts.specialEasing ); + + for ( ; index < length; index++ ) { + result = Animation.prefilters[ index ].call( animation, elem, props, animation.opts ); + if ( result ) { + if ( isFunction( result.stop ) ) { + jQuery._queueHooks( animation.elem, animation.opts.queue ).stop = + result.stop.bind( result ); + } + return result; + } + } + + jQuery.map( props, createTween, animation ); + + if ( isFunction( animation.opts.start ) ) { + animation.opts.start.call( elem, animation ); + } + + // Attach callbacks from options + animation + .progress( animation.opts.progress ) + .done( animation.opts.done, animation.opts.complete ) + .fail( animation.opts.fail ) + .always( animation.opts.always ); + + jQuery.fx.timer( + jQuery.extend( tick, { + elem: elem, + anim: animation, + queue: animation.opts.queue + } ) + ); + + return animation; +} + +jQuery.Animation = jQuery.extend( Animation, { + + tweeners: { + "*": [ function( prop, value ) { + var tween = this.createTween( prop, value ); + adjustCSS( tween.elem, prop, rcssNum.exec( value ), tween ); + return tween; + } ] + }, + + tweener: function( props, callback ) { + if ( isFunction( props ) ) { + callback = props; + props = [ "*" ]; + } else { + props = props.match( rnothtmlwhite ); + } + + var prop, + index = 0, + length = props.length; + + for ( ; index < length; index++ ) { + prop = props[ index ]; + Animation.tweeners[ prop ] = Animation.tweeners[ prop ] || []; + Animation.tweeners[ prop ].unshift( callback ); + } + }, + + prefilters: [ defaultPrefilter ], + + prefilter: function( callback, prepend ) { + if ( prepend ) { + Animation.prefilters.unshift( callback ); + } else { + Animation.prefilters.push( callback ); + } + } +} ); + +jQuery.speed = function( speed, easing, fn ) { + var opt = speed && typeof speed === "object" ? jQuery.extend( {}, speed ) : { + complete: fn || !fn && easing || + isFunction( speed ) && speed, + duration: speed, + easing: fn && easing || easing && !isFunction( easing ) && easing + }; + + // Go to the end state if fx are off + if ( jQuery.fx.off ) { + opt.duration = 0; + + } else { + if ( typeof opt.duration !== "number" ) { + if ( opt.duration in jQuery.fx.speeds ) { + opt.duration = jQuery.fx.speeds[ opt.duration ]; + + } else { + opt.duration = jQuery.fx.speeds._default; + } + } + } + + // Normalize opt.queue - true/undefined/null -> "fx" + if ( opt.queue == null || opt.queue === true ) { + opt.queue = "fx"; + } + + // Queueing + opt.old = opt.complete; + + opt.complete = function() { + if ( isFunction( opt.old ) ) { + opt.old.call( this ); + } + + if ( opt.queue ) { + jQuery.dequeue( this, opt.queue ); + } + }; + + return opt; +}; + +jQuery.fn.extend( { + fadeTo: function( speed, to, easing, callback ) { + + // Show any hidden elements after setting opacity to 0 + return this.filter( isHiddenWithinTree ).css( "opacity", 0 ).show() + + // Animate to the value specified + .end().animate( { opacity: to }, speed, easing, callback ); + }, + animate: function( prop, speed, easing, callback ) { + var empty = jQuery.isEmptyObject( prop ), + optall = jQuery.speed( speed, easing, callback ), + doAnimation = function() { + + // Operate on a copy of prop so per-property easing won't be lost + var anim = Animation( this, jQuery.extend( {}, prop ), optall ); + + // Empty animations, or finishing resolves immediately + if ( empty || dataPriv.get( this, "finish" ) ) { + anim.stop( true ); + } + }; + doAnimation.finish = doAnimation; + + return empty || optall.queue === false ? + this.each( doAnimation ) : + this.queue( optall.queue, doAnimation ); + }, + stop: function( type, clearQueue, gotoEnd ) { + var stopQueue = function( hooks ) { + var stop = hooks.stop; + delete hooks.stop; + stop( gotoEnd ); + }; + + if ( typeof type !== "string" ) { + gotoEnd = clearQueue; + clearQueue = type; + type = undefined; + } + if ( clearQueue && type !== false ) { + this.queue( type || "fx", [] ); + } + + return this.each( function() { + var dequeue = true, + index = type != null && type + "queueHooks", + timers = jQuery.timers, + data = dataPriv.get( this ); + + if ( index ) { + if ( data[ index ] && data[ index ].stop ) { + stopQueue( data[ index ] ); + } + } else { + for ( index in data ) { + if ( data[ index ] && data[ index ].stop && rrun.test( index ) ) { + stopQueue( data[ index ] ); + } + } + } + + for ( index = timers.length; index--; ) { + if ( timers[ index ].elem === this && + ( type == null || timers[ index ].queue === type ) ) { + + timers[ index ].anim.stop( gotoEnd ); + dequeue = false; + timers.splice( index, 1 ); + } + } + + // Start the next in the queue if the last step wasn't forced. + // Timers currently will call their complete callbacks, which + // will dequeue but only if they were gotoEnd. + if ( dequeue || !gotoEnd ) { + jQuery.dequeue( this, type ); + } + } ); + }, + finish: function( type ) { + if ( type !== false ) { + type = type || "fx"; + } + return this.each( function() { + var index, + data = dataPriv.get( this ), + queue = data[ type + "queue" ], + hooks = data[ type + "queueHooks" ], + timers = jQuery.timers, + length = queue ? queue.length : 0; + + // Enable finishing flag on private data + data.finish = true; + + // Empty the queue first + jQuery.queue( this, type, [] ); + + if ( hooks && hooks.stop ) { + hooks.stop.call( this, true ); + } + + // Look for any active animations, and finish them + for ( index = timers.length; index--; ) { + if ( timers[ index ].elem === this && timers[ index ].queue === type ) { + timers[ index ].anim.stop( true ); + timers.splice( index, 1 ); + } + } + + // Look for any animations in the old queue and finish them + for ( index = 0; index < length; index++ ) { + if ( queue[ index ] && queue[ index ].finish ) { + queue[ index ].finish.call( this ); + } + } + + // Turn off finishing flag + delete data.finish; + } ); + } +} ); + +jQuery.each( [ "toggle", "show", "hide" ], function( i, name ) { + var cssFn = jQuery.fn[ name ]; + jQuery.fn[ name ] = function( speed, easing, callback ) { + return speed == null || typeof speed === "boolean" ? + cssFn.apply( this, arguments ) : + this.animate( genFx( name, true ), speed, easing, callback ); + }; +} ); + +// Generate shortcuts for custom animations +jQuery.each( { + slideDown: genFx( "show" ), + slideUp: genFx( "hide" ), + slideToggle: genFx( "toggle" ), + fadeIn: { opacity: "show" }, + fadeOut: { opacity: "hide" }, + fadeToggle: { opacity: "toggle" } +}, function( name, props ) { + jQuery.fn[ name ] = function( speed, easing, callback ) { + return this.animate( props, speed, easing, callback ); + }; +} ); + +jQuery.timers = []; +jQuery.fx.tick = function() { + var timer, + i = 0, + timers = jQuery.timers; + + fxNow = Date.now(); + + for ( ; i < timers.length; i++ ) { + timer = timers[ i ]; + + // Run the timer and safely remove it when done (allowing for external removal) + if ( !timer() && timers[ i ] === timer ) { + timers.splice( i--, 1 ); + } + } + + if ( !timers.length ) { + jQuery.fx.stop(); + } + fxNow = undefined; +}; + +jQuery.fx.timer = function( timer ) { + jQuery.timers.push( timer ); + jQuery.fx.start(); +}; + +jQuery.fx.interval = 13; +jQuery.fx.start = function() { + if ( inProgress ) { + return; + } + + inProgress = true; + schedule(); +}; + +jQuery.fx.stop = function() { + inProgress = null; +}; + +jQuery.fx.speeds = { + slow: 600, + fast: 200, + + // Default speed + _default: 400 +}; + + +// Based off of the plugin by Clint Helfers, with permission. +// https://web.archive.org/web/20100324014747/http://blindsignals.com/index.php/2009/07/jquery-delay/ +jQuery.fn.delay = function( time, type ) { + time = jQuery.fx ? jQuery.fx.speeds[ time ] || time : time; + type = type || "fx"; + + return this.queue( type, function( next, hooks ) { + var timeout = window.setTimeout( next, time ); + hooks.stop = function() { + window.clearTimeout( timeout ); + }; + } ); +}; + + +( function() { + var input = document.createElement( "input" ), + select = document.createElement( "select" ), + opt = select.appendChild( document.createElement( "option" ) ); + + input.type = "checkbox"; + + // Support: Android <=4.3 only + // Default value for a checkbox should be "on" + support.checkOn = input.value !== ""; + + // Support: IE <=11 only + // Must access selectedIndex to make default options select + support.optSelected = opt.selected; + + // Support: IE <=11 only + // An input loses its value after becoming a radio + input = document.createElement( "input" ); + input.value = "t"; + input.type = "radio"; + support.radioValue = input.value === "t"; +} )(); + + +var boolHook, + attrHandle = jQuery.expr.attrHandle; + +jQuery.fn.extend( { + attr: function( name, value ) { + return access( this, jQuery.attr, name, value, arguments.length > 1 ); + }, + + removeAttr: function( name ) { + return this.each( function() { + jQuery.removeAttr( this, name ); + } ); + } +} ); + +jQuery.extend( { + attr: function( elem, name, value ) { + var ret, hooks, + nType = elem.nodeType; + + // Don't get/set attributes on text, comment and attribute nodes + if ( nType === 3 || nType === 8 || nType === 2 ) { + return; + } + + // Fallback to prop when attributes are not supported + if ( typeof elem.getAttribute === "undefined" ) { + return jQuery.prop( elem, name, value ); + } + + // Attribute hooks are determined by the lowercase version + // Grab necessary hook if one is defined + if ( nType !== 1 || !jQuery.isXMLDoc( elem ) ) { + hooks = jQuery.attrHooks[ name.toLowerCase() ] || + ( jQuery.expr.match.bool.test( name ) ? boolHook : undefined ); + } + + if ( value !== undefined ) { + if ( value === null ) { + jQuery.removeAttr( elem, name ); + return; + } + + if ( hooks && "set" in hooks && + ( ret = hooks.set( elem, value, name ) ) !== undefined ) { + return ret; + } + + elem.setAttribute( name, value + "" ); + return value; + } + + if ( hooks && "get" in hooks && ( ret = hooks.get( elem, name ) ) !== null ) { + return ret; + } + + ret = jQuery.find.attr( elem, name ); + + // Non-existent attributes return null, we normalize to undefined + return ret == null ? undefined : ret; + }, + + attrHooks: { + type: { + set: function( elem, value ) { + if ( !support.radioValue && value === "radio" && + nodeName( elem, "input" ) ) { + var val = elem.value; + elem.setAttribute( "type", value ); + if ( val ) { + elem.value = val; + } + return value; + } + } + } + }, + + removeAttr: function( elem, value ) { + var name, + i = 0, + + // Attribute names can contain non-HTML whitespace characters + // https://html.spec.whatwg.org/multipage/syntax.html#attributes-2 + attrNames = value && value.match( rnothtmlwhite ); + + if ( attrNames && elem.nodeType === 1 ) { + while ( ( name = attrNames[ i++ ] ) ) { + elem.removeAttribute( name ); + } + } + } +} ); + +// Hooks for boolean attributes +boolHook = { + set: function( elem, value, name ) { + if ( value === false ) { + + // Remove boolean attributes when set to false + jQuery.removeAttr( elem, name ); + } else { + elem.setAttribute( name, name ); + } + return name; + } +}; + +jQuery.each( jQuery.expr.match.bool.source.match( /\w+/g ), function( i, name ) { + var getter = attrHandle[ name ] || jQuery.find.attr; + + attrHandle[ name ] = function( elem, name, isXML ) { + var ret, handle, + lowercaseName = name.toLowerCase(); + + if ( !isXML ) { + + // Avoid an infinite loop by temporarily removing this function from the getter + handle = attrHandle[ lowercaseName ]; + attrHandle[ lowercaseName ] = ret; + ret = getter( elem, name, isXML ) != null ? + lowercaseName : + null; + attrHandle[ lowercaseName ] = handle; + } + return ret; + }; +} ); + + + + +var rfocusable = /^(?:input|select|textarea|button)$/i, + rclickable = /^(?:a|area)$/i; + +jQuery.fn.extend( { + prop: function( name, value ) { + return access( this, jQuery.prop, name, value, arguments.length > 1 ); + }, + + removeProp: function( name ) { + return this.each( function() { + delete this[ jQuery.propFix[ name ] || name ]; + } ); + } +} ); + +jQuery.extend( { + prop: function( elem, name, value ) { + var ret, hooks, + nType = elem.nodeType; + + // Don't get/set properties on text, comment and attribute nodes + if ( nType === 3 || nType === 8 || nType === 2 ) { + return; + } + + if ( nType !== 1 || !jQuery.isXMLDoc( elem ) ) { + + // Fix name and attach hooks + name = jQuery.propFix[ name ] || name; + hooks = jQuery.propHooks[ name ]; + } + + if ( value !== undefined ) { + if ( hooks && "set" in hooks && + ( ret = hooks.set( elem, value, name ) ) !== undefined ) { + return ret; + } + + return ( elem[ name ] = value ); + } + + if ( hooks && "get" in hooks && ( ret = hooks.get( elem, name ) ) !== null ) { + return ret; + } + + return elem[ name ]; + }, + + propHooks: { + tabIndex: { + get: function( elem ) { + + // Support: IE <=9 - 11 only + // elem.tabIndex doesn't always return the + // correct value when it hasn't been explicitly set + // https://web.archive.org/web/20141116233347/http://fluidproject.org/blog/2008/01/09/getting-setting-and-removing-tabindex-values-with-javascript/ + // Use proper attribute retrieval(#12072) + var tabindex = jQuery.find.attr( elem, "tabindex" ); + + if ( tabindex ) { + return parseInt( tabindex, 10 ); + } + + if ( + rfocusable.test( elem.nodeName ) || + rclickable.test( elem.nodeName ) && + elem.href + ) { + return 0; + } + + return -1; + } + } + }, + + propFix: { + "for": "htmlFor", + "class": "className" + } +} ); + +// Support: IE <=11 only +// Accessing the selectedIndex property +// forces the browser to respect setting selected +// on the option +// The getter ensures a default option is selected +// when in an optgroup +// eslint rule "no-unused-expressions" is disabled for this code +// since it considers such accessions noop +if ( !support.optSelected ) { + jQuery.propHooks.selected = { + get: function( elem ) { + + /* eslint no-unused-expressions: "off" */ + + var parent = elem.parentNode; + if ( parent && parent.parentNode ) { + parent.parentNode.selectedIndex; + } + return null; + }, + set: function( elem ) { + + /* eslint no-unused-expressions: "off" */ + + var parent = elem.parentNode; + if ( parent ) { + parent.selectedIndex; + + if ( parent.parentNode ) { + parent.parentNode.selectedIndex; + } + } + } + }; +} + +jQuery.each( [ + "tabIndex", + "readOnly", + "maxLength", + "cellSpacing", + "cellPadding", + "rowSpan", + "colSpan", + "useMap", + "frameBorder", + "contentEditable" +], function() { + jQuery.propFix[ this.toLowerCase() ] = this; +} ); + + + + + // Strip and collapse whitespace according to HTML spec + // https://infra.spec.whatwg.org/#strip-and-collapse-ascii-whitespace + function stripAndCollapse( value ) { + var tokens = value.match( rnothtmlwhite ) || []; + return tokens.join( " " ); + } + + +function getClass( elem ) { + return elem.getAttribute && elem.getAttribute( "class" ) || ""; +} + +function classesToArray( value ) { + if ( Array.isArray( value ) ) { + return value; + } + if ( typeof value === "string" ) { + return value.match( rnothtmlwhite ) || []; + } + return []; +} + +jQuery.fn.extend( { + addClass: function( value ) { + var classes, elem, cur, curValue, clazz, j, finalValue, + i = 0; + + if ( isFunction( value ) ) { + return this.each( function( j ) { + jQuery( this ).addClass( value.call( this, j, getClass( this ) ) ); + } ); + } + + classes = classesToArray( value ); + + if ( classes.length ) { + while ( ( elem = this[ i++ ] ) ) { + curValue = getClass( elem ); + cur = elem.nodeType === 1 && ( " " + stripAndCollapse( curValue ) + " " ); + + if ( cur ) { + j = 0; + while ( ( clazz = classes[ j++ ] ) ) { + if ( cur.indexOf( " " + clazz + " " ) < 0 ) { + cur += clazz + " "; + } + } + + // Only assign if different to avoid unneeded rendering. + finalValue = stripAndCollapse( cur ); + if ( curValue !== finalValue ) { + elem.setAttribute( "class", finalValue ); + } + } + } + } + + return this; + }, + + removeClass: function( value ) { + var classes, elem, cur, curValue, clazz, j, finalValue, + i = 0; + + if ( isFunction( value ) ) { + return this.each( function( j ) { + jQuery( this ).removeClass( value.call( this, j, getClass( this ) ) ); + } ); + } + + if ( !arguments.length ) { + return this.attr( "class", "" ); + } + + classes = classesToArray( value ); + + if ( classes.length ) { + while ( ( elem = this[ i++ ] ) ) { + curValue = getClass( elem ); + + // This expression is here for better compressibility (see addClass) + cur = elem.nodeType === 1 && ( " " + stripAndCollapse( curValue ) + " " ); + + if ( cur ) { + j = 0; + while ( ( clazz = classes[ j++ ] ) ) { + + // Remove *all* instances + while ( cur.indexOf( " " + clazz + " " ) > -1 ) { + cur = cur.replace( " " + clazz + " ", " " ); + } + } + + // Only assign if different to avoid unneeded rendering. + finalValue = stripAndCollapse( cur ); + if ( curValue !== finalValue ) { + elem.setAttribute( "class", finalValue ); + } + } + } + } + + return this; + }, + + toggleClass: function( value, stateVal ) { + var type = typeof value, + isValidValue = type === "string" || Array.isArray( value ); + + if ( typeof stateVal === "boolean" && isValidValue ) { + return stateVal ? this.addClass( value ) : this.removeClass( value ); + } + + if ( isFunction( value ) ) { + return this.each( function( i ) { + jQuery( this ).toggleClass( + value.call( this, i, getClass( this ), stateVal ), + stateVal + ); + } ); + } + + return this.each( function() { + var className, i, self, classNames; + + if ( isValidValue ) { + + // Toggle individual class names + i = 0; + self = jQuery( this ); + classNames = classesToArray( value ); + + while ( ( className = classNames[ i++ ] ) ) { + + // Check each className given, space separated list + if ( self.hasClass( className ) ) { + self.removeClass( className ); + } else { + self.addClass( className ); + } + } + + // Toggle whole class name + } else if ( value === undefined || type === "boolean" ) { + className = getClass( this ); + if ( className ) { + + // Store className if set + dataPriv.set( this, "__className__", className ); + } + + // If the element has a class name or if we're passed `false`, + // then remove the whole classname (if there was one, the above saved it). + // Otherwise bring back whatever was previously saved (if anything), + // falling back to the empty string if nothing was stored. + if ( this.setAttribute ) { + this.setAttribute( "class", + className || value === false ? + "" : + dataPriv.get( this, "__className__" ) || "" + ); + } + } + } ); + }, + + hasClass: function( selector ) { + var className, elem, + i = 0; + + className = " " + selector + " "; + while ( ( elem = this[ i++ ] ) ) { + if ( elem.nodeType === 1 && + ( " " + stripAndCollapse( getClass( elem ) ) + " " ).indexOf( className ) > -1 ) { + return true; + } + } + + return false; + } +} ); + + + + +var rreturn = /\r/g; + +jQuery.fn.extend( { + val: function( value ) { + var hooks, ret, valueIsFunction, + elem = this[ 0 ]; + + if ( !arguments.length ) { + if ( elem ) { + hooks = jQuery.valHooks[ elem.type ] || + jQuery.valHooks[ elem.nodeName.toLowerCase() ]; + + if ( hooks && + "get" in hooks && + ( ret = hooks.get( elem, "value" ) ) !== undefined + ) { + return ret; + } + + ret = elem.value; + + // Handle most common string cases + if ( typeof ret === "string" ) { + return ret.replace( rreturn, "" ); + } + + // Handle cases where value is null/undef or number + return ret == null ? "" : ret; + } + + return; + } + + valueIsFunction = isFunction( value ); + + return this.each( function( i ) { + var val; + + if ( this.nodeType !== 1 ) { + return; + } + + if ( valueIsFunction ) { + val = value.call( this, i, jQuery( this ).val() ); + } else { + val = value; + } + + // Treat null/undefined as ""; convert numbers to string + if ( val == null ) { + val = ""; + + } else if ( typeof val === "number" ) { + val += ""; + + } else if ( Array.isArray( val ) ) { + val = jQuery.map( val, function( value ) { + return value == null ? "" : value + ""; + } ); + } + + hooks = jQuery.valHooks[ this.type ] || jQuery.valHooks[ this.nodeName.toLowerCase() ]; + + // If set returns undefined, fall back to normal setting + if ( !hooks || !( "set" in hooks ) || hooks.set( this, val, "value" ) === undefined ) { + this.value = val; + } + } ); + } +} ); + +jQuery.extend( { + valHooks: { + option: { + get: function( elem ) { + + var val = jQuery.find.attr( elem, "value" ); + return val != null ? + val : + + // Support: IE <=10 - 11 only + // option.text throws exceptions (#14686, #14858) + // Strip and collapse whitespace + // https://html.spec.whatwg.org/#strip-and-collapse-whitespace + stripAndCollapse( jQuery.text( elem ) ); + } + }, + select: { + get: function( elem ) { + var value, option, i, + options = elem.options, + index = elem.selectedIndex, + one = elem.type === "select-one", + values = one ? null : [], + max = one ? index + 1 : options.length; + + if ( index < 0 ) { + i = max; + + } else { + i = one ? index : 0; + } + + // Loop through all the selected options + for ( ; i < max; i++ ) { + option = options[ i ]; + + // Support: IE <=9 only + // IE8-9 doesn't update selected after form reset (#2551) + if ( ( option.selected || i === index ) && + + // Don't return options that are disabled or in a disabled optgroup + !option.disabled && + ( !option.parentNode.disabled || + !nodeName( option.parentNode, "optgroup" ) ) ) { + + // Get the specific value for the option + value = jQuery( option ).val(); + + // We don't need an array for one selects + if ( one ) { + return value; + } + + // Multi-Selects return an array + values.push( value ); + } + } + + return values; + }, + + set: function( elem, value ) { + var optionSet, option, + options = elem.options, + values = jQuery.makeArray( value ), + i = options.length; + + while ( i-- ) { + option = options[ i ]; + + /* eslint-disable no-cond-assign */ + + if ( option.selected = + jQuery.inArray( jQuery.valHooks.option.get( option ), values ) > -1 + ) { + optionSet = true; + } + + /* eslint-enable no-cond-assign */ + } + + // Force browsers to behave consistently when non-matching value is set + if ( !optionSet ) { + elem.selectedIndex = -1; + } + return values; + } + } + } +} ); + +// Radios and checkboxes getter/setter +jQuery.each( [ "radio", "checkbox" ], function() { + jQuery.valHooks[ this ] = { + set: function( elem, value ) { + if ( Array.isArray( value ) ) { + return ( elem.checked = jQuery.inArray( jQuery( elem ).val(), value ) > -1 ); + } + } + }; + if ( !support.checkOn ) { + jQuery.valHooks[ this ].get = function( elem ) { + return elem.getAttribute( "value" ) === null ? "on" : elem.value; + }; + } +} ); + + + + +// Return jQuery for attributes-only inclusion + + +support.focusin = "onfocusin" in window; + + +var rfocusMorph = /^(?:focusinfocus|focusoutblur)$/, + stopPropagationCallback = function( e ) { + e.stopPropagation(); + }; + +jQuery.extend( jQuery.event, { + + trigger: function( event, data, elem, onlyHandlers ) { + + var i, cur, tmp, bubbleType, ontype, handle, special, lastElement, + eventPath = [ elem || document ], + type = hasOwn.call( event, "type" ) ? event.type : event, + namespaces = hasOwn.call( event, "namespace" ) ? event.namespace.split( "." ) : []; + + cur = lastElement = tmp = elem = elem || document; + + // Don't do events on text and comment nodes + if ( elem.nodeType === 3 || elem.nodeType === 8 ) { + return; + } + + // focus/blur morphs to focusin/out; ensure we're not firing them right now + if ( rfocusMorph.test( type + jQuery.event.triggered ) ) { + return; + } + + if ( type.indexOf( "." ) > -1 ) { + + // Namespaced trigger; create a regexp to match event type in handle() + namespaces = type.split( "." ); + type = namespaces.shift(); + namespaces.sort(); + } + ontype = type.indexOf( ":" ) < 0 && "on" + type; + + // Caller can pass in a jQuery.Event object, Object, or just an event type string + event = event[ jQuery.expando ] ? + event : + new jQuery.Event( type, typeof event === "object" && event ); + + // Trigger bitmask: & 1 for native handlers; & 2 for jQuery (always true) + event.isTrigger = onlyHandlers ? 2 : 3; + event.namespace = namespaces.join( "." ); + event.rnamespace = event.namespace ? + new RegExp( "(^|\\.)" + namespaces.join( "\\.(?:.*\\.|)" ) + "(\\.|$)" ) : + null; + + // Clean up the event in case it is being reused + event.result = undefined; + if ( !event.target ) { + event.target = elem; + } + + // Clone any incoming data and prepend the event, creating the handler arg list + data = data == null ? + [ event ] : + jQuery.makeArray( data, [ event ] ); + + // Allow special events to draw outside the lines + special = jQuery.event.special[ type ] || {}; + if ( !onlyHandlers && special.trigger && special.trigger.apply( elem, data ) === false ) { + return; + } + + // Determine event propagation path in advance, per W3C events spec (#9951) + // Bubble up to document, then to window; watch for a global ownerDocument var (#9724) + if ( !onlyHandlers && !special.noBubble && !isWindow( elem ) ) { + + bubbleType = special.delegateType || type; + if ( !rfocusMorph.test( bubbleType + type ) ) { + cur = cur.parentNode; + } + for ( ; cur; cur = cur.parentNode ) { + eventPath.push( cur ); + tmp = cur; + } + + // Only add window if we got to document (e.g., not plain obj or detached DOM) + if ( tmp === ( elem.ownerDocument || document ) ) { + eventPath.push( tmp.defaultView || tmp.parentWindow || window ); + } + } + + // Fire handlers on the event path + i = 0; + while ( ( cur = eventPath[ i++ ] ) && !event.isPropagationStopped() ) { + lastElement = cur; + event.type = i > 1 ? + bubbleType : + special.bindType || type; + + // jQuery handler + handle = ( dataPriv.get( cur, "events" ) || {} )[ event.type ] && + dataPriv.get( cur, "handle" ); + if ( handle ) { + handle.apply( cur, data ); + } + + // Native handler + handle = ontype && cur[ ontype ]; + if ( handle && handle.apply && acceptData( cur ) ) { + event.result = handle.apply( cur, data ); + if ( event.result === false ) { + event.preventDefault(); + } + } + } + event.type = type; + + // If nobody prevented the default action, do it now + if ( !onlyHandlers && !event.isDefaultPrevented() ) { + + if ( ( !special._default || + special._default.apply( eventPath.pop(), data ) === false ) && + acceptData( elem ) ) { + + // Call a native DOM method on the target with the same name as the event. + // Don't do default actions on window, that's where global variables be (#6170) + if ( ontype && isFunction( elem[ type ] ) && !isWindow( elem ) ) { + + // Don't re-trigger an onFOO event when we call its FOO() method + tmp = elem[ ontype ]; + + if ( tmp ) { + elem[ ontype ] = null; + } + + // Prevent re-triggering of the same event, since we already bubbled it above + jQuery.event.triggered = type; + + if ( event.isPropagationStopped() ) { + lastElement.addEventListener( type, stopPropagationCallback ); + } + + elem[ type ](); + + if ( event.isPropagationStopped() ) { + lastElement.removeEventListener( type, stopPropagationCallback ); + } + + jQuery.event.triggered = undefined; + + if ( tmp ) { + elem[ ontype ] = tmp; + } + } + } + } + + return event.result; + }, + + // Piggyback on a donor event to simulate a different one + // Used only for `focus(in | out)` events + simulate: function( type, elem, event ) { + var e = jQuery.extend( + new jQuery.Event(), + event, + { + type: type, + isSimulated: true + } + ); + + jQuery.event.trigger( e, null, elem ); + } + +} ); + +jQuery.fn.extend( { + + trigger: function( type, data ) { + return this.each( function() { + jQuery.event.trigger( type, data, this ); + } ); + }, + triggerHandler: function( type, data ) { + var elem = this[ 0 ]; + if ( elem ) { + return jQuery.event.trigger( type, data, elem, true ); + } + } +} ); + + +// Support: Firefox <=44 +// Firefox doesn't have focus(in | out) events +// Related ticket - https://bugzilla.mozilla.org/show_bug.cgi?id=687787 +// +// Support: Chrome <=48 - 49, Safari <=9.0 - 9.1 +// focus(in | out) events fire after focus & blur events, +// which is spec violation - http://www.w3.org/TR/DOM-Level-3-Events/#events-focusevent-event-order +// Related ticket - https://bugs.chromium.org/p/chromium/issues/detail?id=449857 +if ( !support.focusin ) { + jQuery.each( { focus: "focusin", blur: "focusout" }, function( orig, fix ) { + + // Attach a single capturing handler on the document while someone wants focusin/focusout + var handler = function( event ) { + jQuery.event.simulate( fix, event.target, jQuery.event.fix( event ) ); + }; + + jQuery.event.special[ fix ] = { + setup: function() { + var doc = this.ownerDocument || this, + attaches = dataPriv.access( doc, fix ); + + if ( !attaches ) { + doc.addEventListener( orig, handler, true ); + } + dataPriv.access( doc, fix, ( attaches || 0 ) + 1 ); + }, + teardown: function() { + var doc = this.ownerDocument || this, + attaches = dataPriv.access( doc, fix ) - 1; + + if ( !attaches ) { + doc.removeEventListener( orig, handler, true ); + dataPriv.remove( doc, fix ); + + } else { + dataPriv.access( doc, fix, attaches ); + } + } + }; + } ); +} +var location = window.location; + +var nonce = Date.now(); + +var rquery = ( /\?/ ); + + + +// Cross-browser xml parsing +jQuery.parseXML = function( data ) { + var xml; + if ( !data || typeof data !== "string" ) { + return null; + } + + // Support: IE 9 - 11 only + // IE throws on parseFromString with invalid input. + try { + xml = ( new window.DOMParser() ).parseFromString( data, "text/xml" ); + } catch ( e ) { + xml = undefined; + } + + if ( !xml || xml.getElementsByTagName( "parsererror" ).length ) { + jQuery.error( "Invalid XML: " + data ); + } + return xml; +}; + + +var + rbracket = /\[\]$/, + rCRLF = /\r?\n/g, + rsubmitterTypes = /^(?:submit|button|image|reset|file)$/i, + rsubmittable = /^(?:input|select|textarea|keygen)/i; + +function buildParams( prefix, obj, traditional, add ) { + var name; + + if ( Array.isArray( obj ) ) { + + // Serialize array item. + jQuery.each( obj, function( i, v ) { + if ( traditional || rbracket.test( prefix ) ) { + + // Treat each array item as a scalar. + add( prefix, v ); + + } else { + + // Item is non-scalar (array or object), encode its numeric index. + buildParams( + prefix + "[" + ( typeof v === "object" && v != null ? i : "" ) + "]", + v, + traditional, + add + ); + } + } ); + + } else if ( !traditional && toType( obj ) === "object" ) { + + // Serialize object item. + for ( name in obj ) { + buildParams( prefix + "[" + name + "]", obj[ name ], traditional, add ); + } + + } else { + + // Serialize scalar item. + add( prefix, obj ); + } +} + +// Serialize an array of form elements or a set of +// key/values into a query string +jQuery.param = function( a, traditional ) { + var prefix, + s = [], + add = function( key, valueOrFunction ) { + + // If value is a function, invoke it and use its return value + var value = isFunction( valueOrFunction ) ? + valueOrFunction() : + valueOrFunction; + + s[ s.length ] = encodeURIComponent( key ) + "=" + + encodeURIComponent( value == null ? "" : value ); + }; + + if ( a == null ) { + return ""; + } + + // If an array was passed in, assume that it is an array of form elements. + if ( Array.isArray( a ) || ( a.jquery && !jQuery.isPlainObject( a ) ) ) { + + // Serialize the form elements + jQuery.each( a, function() { + add( this.name, this.value ); + } ); + + } else { + + // If traditional, encode the "old" way (the way 1.3.2 or older + // did it), otherwise encode params recursively. + for ( prefix in a ) { + buildParams( prefix, a[ prefix ], traditional, add ); + } + } + + // Return the resulting serialization + return s.join( "&" ); +}; + +jQuery.fn.extend( { + serialize: function() { + return jQuery.param( this.serializeArray() ); + }, + serializeArray: function() { + return this.map( function() { + + // Can add propHook for "elements" to filter or add form elements + var elements = jQuery.prop( this, "elements" ); + return elements ? jQuery.makeArray( elements ) : this; + } ) + .filter( function() { + var type = this.type; + + // Use .is( ":disabled" ) so that fieldset[disabled] works + return this.name && !jQuery( this ).is( ":disabled" ) && + rsubmittable.test( this.nodeName ) && !rsubmitterTypes.test( type ) && + ( this.checked || !rcheckableType.test( type ) ); + } ) + .map( function( i, elem ) { + var val = jQuery( this ).val(); + + if ( val == null ) { + return null; + } + + if ( Array.isArray( val ) ) { + return jQuery.map( val, function( val ) { + return { name: elem.name, value: val.replace( rCRLF, "\r\n" ) }; + } ); + } + + return { name: elem.name, value: val.replace( rCRLF, "\r\n" ) }; + } ).get(); + } +} ); + + +var + r20 = /%20/g, + rhash = /#.*$/, + rantiCache = /([?&])_=[^&]*/, + rheaders = /^(.*?):[ \t]*([^\r\n]*)$/mg, + + // #7653, #8125, #8152: local protocol detection + rlocalProtocol = /^(?:about|app|app-storage|.+-extension|file|res|widget):$/, + rnoContent = /^(?:GET|HEAD)$/, + rprotocol = /^\/\//, + + /* Prefilters + * 1) They are useful to introduce custom dataTypes (see ajax/jsonp.js for an example) + * 2) These are called: + * - BEFORE asking for a transport + * - AFTER param serialization (s.data is a string if s.processData is true) + * 3) key is the dataType + * 4) the catchall symbol "*" can be used + * 5) execution will start with transport dataType and THEN continue down to "*" if needed + */ + prefilters = {}, + + /* Transports bindings + * 1) key is the dataType + * 2) the catchall symbol "*" can be used + * 3) selection will start with transport dataType and THEN go to "*" if needed + */ + transports = {}, + + // Avoid comment-prolog char sequence (#10098); must appease lint and evade compression + allTypes = "*/".concat( "*" ), + + // Anchor tag for parsing the document origin + originAnchor = document.createElement( "a" ); + originAnchor.href = location.href; + +// Base "constructor" for jQuery.ajaxPrefilter and jQuery.ajaxTransport +function addToPrefiltersOrTransports( structure ) { + + // dataTypeExpression is optional and defaults to "*" + return function( dataTypeExpression, func ) { + + if ( typeof dataTypeExpression !== "string" ) { + func = dataTypeExpression; + dataTypeExpression = "*"; + } + + var dataType, + i = 0, + dataTypes = dataTypeExpression.toLowerCase().match( rnothtmlwhite ) || []; + + if ( isFunction( func ) ) { + + // For each dataType in the dataTypeExpression + while ( ( dataType = dataTypes[ i++ ] ) ) { + + // Prepend if requested + if ( dataType[ 0 ] === "+" ) { + dataType = dataType.slice( 1 ) || "*"; + ( structure[ dataType ] = structure[ dataType ] || [] ).unshift( func ); + + // Otherwise append + } else { + ( structure[ dataType ] = structure[ dataType ] || [] ).push( func ); + } + } + } + }; +} + +// Base inspection function for prefilters and transports +function inspectPrefiltersOrTransports( structure, options, originalOptions, jqXHR ) { + + var inspected = {}, + seekingTransport = ( structure === transports ); + + function inspect( dataType ) { + var selected; + inspected[ dataType ] = true; + jQuery.each( structure[ dataType ] || [], function( _, prefilterOrFactory ) { + var dataTypeOrTransport = prefilterOrFactory( options, originalOptions, jqXHR ); + if ( typeof dataTypeOrTransport === "string" && + !seekingTransport && !inspected[ dataTypeOrTransport ] ) { + + options.dataTypes.unshift( dataTypeOrTransport ); + inspect( dataTypeOrTransport ); + return false; + } else if ( seekingTransport ) { + return !( selected = dataTypeOrTransport ); + } + } ); + return selected; + } + + return inspect( options.dataTypes[ 0 ] ) || !inspected[ "*" ] && inspect( "*" ); +} + +// A special extend for ajax options +// that takes "flat" options (not to be deep extended) +// Fixes #9887 +function ajaxExtend( target, src ) { + var key, deep, + flatOptions = jQuery.ajaxSettings.flatOptions || {}; + + for ( key in src ) { + if ( src[ key ] !== undefined ) { + ( flatOptions[ key ] ? target : ( deep || ( deep = {} ) ) )[ key ] = src[ key ]; + } + } + if ( deep ) { + jQuery.extend( true, target, deep ); + } + + return target; +} + +/* Handles responses to an ajax request: + * - finds the right dataType (mediates between content-type and expected dataType) + * - returns the corresponding response + */ +function ajaxHandleResponses( s, jqXHR, responses ) { + + var ct, type, finalDataType, firstDataType, + contents = s.contents, + dataTypes = s.dataTypes; + + // Remove auto dataType and get content-type in the process + while ( dataTypes[ 0 ] === "*" ) { + dataTypes.shift(); + if ( ct === undefined ) { + ct = s.mimeType || jqXHR.getResponseHeader( "Content-Type" ); + } + } + + // Check if we're dealing with a known content-type + if ( ct ) { + for ( type in contents ) { + if ( contents[ type ] && contents[ type ].test( ct ) ) { + dataTypes.unshift( type ); + break; + } + } + } + + // Check to see if we have a response for the expected dataType + if ( dataTypes[ 0 ] in responses ) { + finalDataType = dataTypes[ 0 ]; + } else { + + // Try convertible dataTypes + for ( type in responses ) { + if ( !dataTypes[ 0 ] || s.converters[ type + " " + dataTypes[ 0 ] ] ) { + finalDataType = type; + break; + } + if ( !firstDataType ) { + firstDataType = type; + } + } + + // Or just use first one + finalDataType = finalDataType || firstDataType; + } + + // If we found a dataType + // We add the dataType to the list if needed + // and return the corresponding response + if ( finalDataType ) { + if ( finalDataType !== dataTypes[ 0 ] ) { + dataTypes.unshift( finalDataType ); + } + return responses[ finalDataType ]; + } +} + +/* Chain conversions given the request and the original response + * Also sets the responseXXX fields on the jqXHR instance + */ +function ajaxConvert( s, response, jqXHR, isSuccess ) { + var conv2, current, conv, tmp, prev, + converters = {}, + + // Work with a copy of dataTypes in case we need to modify it for conversion + dataTypes = s.dataTypes.slice(); + + // Create converters map with lowercased keys + if ( dataTypes[ 1 ] ) { + for ( conv in s.converters ) { + converters[ conv.toLowerCase() ] = s.converters[ conv ]; + } + } + + current = dataTypes.shift(); + + // Convert to each sequential dataType + while ( current ) { + + if ( s.responseFields[ current ] ) { + jqXHR[ s.responseFields[ current ] ] = response; + } + + // Apply the dataFilter if provided + if ( !prev && isSuccess && s.dataFilter ) { + response = s.dataFilter( response, s.dataType ); + } + + prev = current; + current = dataTypes.shift(); + + if ( current ) { + + // There's only work to do if current dataType is non-auto + if ( current === "*" ) { + + current = prev; + + // Convert response if prev dataType is non-auto and differs from current + } else if ( prev !== "*" && prev !== current ) { + + // Seek a direct converter + conv = converters[ prev + " " + current ] || converters[ "* " + current ]; + + // If none found, seek a pair + if ( !conv ) { + for ( conv2 in converters ) { + + // If conv2 outputs current + tmp = conv2.split( " " ); + if ( tmp[ 1 ] === current ) { + + // If prev can be converted to accepted input + conv = converters[ prev + " " + tmp[ 0 ] ] || + converters[ "* " + tmp[ 0 ] ]; + if ( conv ) { + + // Condense equivalence converters + if ( conv === true ) { + conv = converters[ conv2 ]; + + // Otherwise, insert the intermediate dataType + } else if ( converters[ conv2 ] !== true ) { + current = tmp[ 0 ]; + dataTypes.unshift( tmp[ 1 ] ); + } + break; + } + } + } + } + + // Apply converter (if not an equivalence) + if ( conv !== true ) { + + // Unless errors are allowed to bubble, catch and return them + if ( conv && s.throws ) { + response = conv( response ); + } else { + try { + response = conv( response ); + } catch ( e ) { + return { + state: "parsererror", + error: conv ? e : "No conversion from " + prev + " to " + current + }; + } + } + } + } + } + } + + return { state: "success", data: response }; +} + +jQuery.extend( { + + // Counter for holding the number of active queries + active: 0, + + // Last-Modified header cache for next request + lastModified: {}, + etag: {}, + + ajaxSettings: { + url: location.href, + type: "GET", + isLocal: rlocalProtocol.test( location.protocol ), + global: true, + processData: true, + async: true, + contentType: "application/x-www-form-urlencoded; charset=UTF-8", + + /* + timeout: 0, + data: null, + dataType: null, + username: null, + password: null, + cache: null, + throws: false, + traditional: false, + headers: {}, + */ + + accepts: { + "*": allTypes, + text: "text/plain", + html: "text/html", + xml: "application/xml, text/xml", + json: "application/json, text/javascript" + }, + + contents: { + xml: /\bxml\b/, + html: /\bhtml/, + json: /\bjson\b/ + }, + + responseFields: { + xml: "responseXML", + text: "responseText", + json: "responseJSON" + }, + + // Data converters + // Keys separate source (or catchall "*") and destination types with a single space + converters: { + + // Convert anything to text + "* text": String, + + // Text to html (true = no transformation) + "text html": true, + + // Evaluate text as a json expression + "text json": JSON.parse, + + // Parse text as xml + "text xml": jQuery.parseXML + }, + + // For options that shouldn't be deep extended: + // you can add your own custom options here if + // and when you create one that shouldn't be + // deep extended (see ajaxExtend) + flatOptions: { + url: true, + context: true + } + }, + + // Creates a full fledged settings object into target + // with both ajaxSettings and settings fields. + // If target is omitted, writes into ajaxSettings. + ajaxSetup: function( target, settings ) { + return settings ? + + // Building a settings object + ajaxExtend( ajaxExtend( target, jQuery.ajaxSettings ), settings ) : + + // Extending ajaxSettings + ajaxExtend( jQuery.ajaxSettings, target ); + }, + + ajaxPrefilter: addToPrefiltersOrTransports( prefilters ), + ajaxTransport: addToPrefiltersOrTransports( transports ), + + // Main method + ajax: function( url, options ) { + + // If url is an object, simulate pre-1.5 signature + if ( typeof url === "object" ) { + options = url; + url = undefined; + } + + // Force options to be an object + options = options || {}; + + var transport, + + // URL without anti-cache param + cacheURL, + + // Response headers + responseHeadersString, + responseHeaders, + + // timeout handle + timeoutTimer, + + // Url cleanup var + urlAnchor, + + // Request state (becomes false upon send and true upon completion) + completed, + + // To know if global events are to be dispatched + fireGlobals, + + // Loop variable + i, + + // uncached part of the url + uncached, + + // Create the final options object + s = jQuery.ajaxSetup( {}, options ), + + // Callbacks context + callbackContext = s.context || s, + + // Context for global events is callbackContext if it is a DOM node or jQuery collection + globalEventContext = s.context && + ( callbackContext.nodeType || callbackContext.jquery ) ? + jQuery( callbackContext ) : + jQuery.event, + + // Deferreds + deferred = jQuery.Deferred(), + completeDeferred = jQuery.Callbacks( "once memory" ), + + // Status-dependent callbacks + statusCode = s.statusCode || {}, + + // Headers (they are sent all at once) + requestHeaders = {}, + requestHeadersNames = {}, + + // Default abort message + strAbort = "canceled", + + // Fake xhr + jqXHR = { + readyState: 0, + + // Builds headers hashtable if needed + getResponseHeader: function( key ) { + var match; + if ( completed ) { + if ( !responseHeaders ) { + responseHeaders = {}; + while ( ( match = rheaders.exec( responseHeadersString ) ) ) { + responseHeaders[ match[ 1 ].toLowerCase() + " " ] = + ( responseHeaders[ match[ 1 ].toLowerCase() + " " ] || [] ) + .concat( match[ 2 ] ); + } + } + match = responseHeaders[ key.toLowerCase() + " " ]; + } + return match == null ? null : match.join( ", " ); + }, + + // Raw string + getAllResponseHeaders: function() { + return completed ? responseHeadersString : null; + }, + + // Caches the header + setRequestHeader: function( name, value ) { + if ( completed == null ) { + name = requestHeadersNames[ name.toLowerCase() ] = + requestHeadersNames[ name.toLowerCase() ] || name; + requestHeaders[ name ] = value; + } + return this; + }, + + // Overrides response content-type header + overrideMimeType: function( type ) { + if ( completed == null ) { + s.mimeType = type; + } + return this; + }, + + // Status-dependent callbacks + statusCode: function( map ) { + var code; + if ( map ) { + if ( completed ) { + + // Execute the appropriate callbacks + jqXHR.always( map[ jqXHR.status ] ); + } else { + + // Lazy-add the new callbacks in a way that preserves old ones + for ( code in map ) { + statusCode[ code ] = [ statusCode[ code ], map[ code ] ]; + } + } + } + return this; + }, + + // Cancel the request + abort: function( statusText ) { + var finalText = statusText || strAbort; + if ( transport ) { + transport.abort( finalText ); + } + done( 0, finalText ); + return this; + } + }; + + // Attach deferreds + deferred.promise( jqXHR ); + + // Add protocol if not provided (prefilters might expect it) + // Handle falsy url in the settings object (#10093: consistency with old signature) + // We also use the url parameter if available + s.url = ( ( url || s.url || location.href ) + "" ) + .replace( rprotocol, location.protocol + "//" ); + + // Alias method option to type as per ticket #12004 + s.type = options.method || options.type || s.method || s.type; + + // Extract dataTypes list + s.dataTypes = ( s.dataType || "*" ).toLowerCase().match( rnothtmlwhite ) || [ "" ]; + + // A cross-domain request is in order when the origin doesn't match the current origin. + if ( s.crossDomain == null ) { + urlAnchor = document.createElement( "a" ); + + // Support: IE <=8 - 11, Edge 12 - 15 + // IE throws exception on accessing the href property if url is malformed, + // e.g. http://example.com:80x/ + try { + urlAnchor.href = s.url; + + // Support: IE <=8 - 11 only + // Anchor's host property isn't correctly set when s.url is relative + urlAnchor.href = urlAnchor.href; + s.crossDomain = originAnchor.protocol + "//" + originAnchor.host !== + urlAnchor.protocol + "//" + urlAnchor.host; + } catch ( e ) { + + // If there is an error parsing the URL, assume it is crossDomain, + // it can be rejected by the transport if it is invalid + s.crossDomain = true; + } + } + + // Convert data if not already a string + if ( s.data && s.processData && typeof s.data !== "string" ) { + s.data = jQuery.param( s.data, s.traditional ); + } + + // Apply prefilters + inspectPrefiltersOrTransports( prefilters, s, options, jqXHR ); + + // If request was aborted inside a prefilter, stop there + if ( completed ) { + return jqXHR; + } + + // We can fire global events as of now if asked to + // Don't fire events if jQuery.event is undefined in an AMD-usage scenario (#15118) + fireGlobals = jQuery.event && s.global; + + // Watch for a new set of requests + if ( fireGlobals && jQuery.active++ === 0 ) { + jQuery.event.trigger( "ajaxStart" ); + } + + // Uppercase the type + s.type = s.type.toUpperCase(); + + // Determine if request has content + s.hasContent = !rnoContent.test( s.type ); + + // Save the URL in case we're toying with the If-Modified-Since + // and/or If-None-Match header later on + // Remove hash to simplify url manipulation + cacheURL = s.url.replace( rhash, "" ); + + // More options handling for requests with no content + if ( !s.hasContent ) { + + // Remember the hash so we can put it back + uncached = s.url.slice( cacheURL.length ); + + // If data is available and should be processed, append data to url + if ( s.data && ( s.processData || typeof s.data === "string" ) ) { + cacheURL += ( rquery.test( cacheURL ) ? "&" : "?" ) + s.data; + + // #9682: remove data so that it's not used in an eventual retry + delete s.data; + } + + // Add or update anti-cache param if needed + if ( s.cache === false ) { + cacheURL = cacheURL.replace( rantiCache, "$1" ); + uncached = ( rquery.test( cacheURL ) ? "&" : "?" ) + "_=" + ( nonce++ ) + uncached; + } + + // Put hash and anti-cache on the URL that will be requested (gh-1732) + s.url = cacheURL + uncached; + + // Change '%20' to '+' if this is encoded form body content (gh-2658) + } else if ( s.data && s.processData && + ( s.contentType || "" ).indexOf( "application/x-www-form-urlencoded" ) === 0 ) { + s.data = s.data.replace( r20, "+" ); + } + + // Set the If-Modified-Since and/or If-None-Match header, if in ifModified mode. + if ( s.ifModified ) { + if ( jQuery.lastModified[ cacheURL ] ) { + jqXHR.setRequestHeader( "If-Modified-Since", jQuery.lastModified[ cacheURL ] ); + } + if ( jQuery.etag[ cacheURL ] ) { + jqXHR.setRequestHeader( "If-None-Match", jQuery.etag[ cacheURL ] ); + } + } + + // Set the correct header, if data is being sent + if ( s.data && s.hasContent && s.contentType !== false || options.contentType ) { + jqXHR.setRequestHeader( "Content-Type", s.contentType ); + } + + // Set the Accepts header for the server, depending on the dataType + jqXHR.setRequestHeader( + "Accept", + s.dataTypes[ 0 ] && s.accepts[ s.dataTypes[ 0 ] ] ? + s.accepts[ s.dataTypes[ 0 ] ] + + ( s.dataTypes[ 0 ] !== "*" ? ", " + allTypes + "; q=0.01" : "" ) : + s.accepts[ "*" ] + ); + + // Check for headers option + for ( i in s.headers ) { + jqXHR.setRequestHeader( i, s.headers[ i ] ); + } + + // Allow custom headers/mimetypes and early abort + if ( s.beforeSend && + ( s.beforeSend.call( callbackContext, jqXHR, s ) === false || completed ) ) { + + // Abort if not done already and return + return jqXHR.abort(); + } + + // Aborting is no longer a cancellation + strAbort = "abort"; + + // Install callbacks on deferreds + completeDeferred.add( s.complete ); + jqXHR.done( s.success ); + jqXHR.fail( s.error ); + + // Get transport + transport = inspectPrefiltersOrTransports( transports, s, options, jqXHR ); + + // If no transport, we auto-abort + if ( !transport ) { + done( -1, "No Transport" ); + } else { + jqXHR.readyState = 1; + + // Send global event + if ( fireGlobals ) { + globalEventContext.trigger( "ajaxSend", [ jqXHR, s ] ); + } + + // If request was aborted inside ajaxSend, stop there + if ( completed ) { + return jqXHR; + } + + // Timeout + if ( s.async && s.timeout > 0 ) { + timeoutTimer = window.setTimeout( function() { + jqXHR.abort( "timeout" ); + }, s.timeout ); + } + + try { + completed = false; + transport.send( requestHeaders, done ); + } catch ( e ) { + + // Rethrow post-completion exceptions + if ( completed ) { + throw e; + } + + // Propagate others as results + done( -1, e ); + } + } + + // Callback for when everything is done + function done( status, nativeStatusText, responses, headers ) { + var isSuccess, success, error, response, modified, + statusText = nativeStatusText; + + // Ignore repeat invocations + if ( completed ) { + return; + } + + completed = true; + + // Clear timeout if it exists + if ( timeoutTimer ) { + window.clearTimeout( timeoutTimer ); + } + + // Dereference transport for early garbage collection + // (no matter how long the jqXHR object will be used) + transport = undefined; + + // Cache response headers + responseHeadersString = headers || ""; + + // Set readyState + jqXHR.readyState = status > 0 ? 4 : 0; + + // Determine if successful + isSuccess = status >= 200 && status < 300 || status === 304; + + // Get response data + if ( responses ) { + response = ajaxHandleResponses( s, jqXHR, responses ); + } + + // Convert no matter what (that way responseXXX fields are always set) + response = ajaxConvert( s, response, jqXHR, isSuccess ); + + // If successful, handle type chaining + if ( isSuccess ) { + + // Set the If-Modified-Since and/or If-None-Match header, if in ifModified mode. + if ( s.ifModified ) { + modified = jqXHR.getResponseHeader( "Last-Modified" ); + if ( modified ) { + jQuery.lastModified[ cacheURL ] = modified; + } + modified = jqXHR.getResponseHeader( "etag" ); + if ( modified ) { + jQuery.etag[ cacheURL ] = modified; + } + } + + // if no content + if ( status === 204 || s.type === "HEAD" ) { + statusText = "nocontent"; + + // if not modified + } else if ( status === 304 ) { + statusText = "notmodified"; + + // If we have data, let's convert it + } else { + statusText = response.state; + success = response.data; + error = response.error; + isSuccess = !error; + } + } else { + + // Extract error from statusText and normalize for non-aborts + error = statusText; + if ( status || !statusText ) { + statusText = "error"; + if ( status < 0 ) { + status = 0; + } + } + } + + // Set data for the fake xhr object + jqXHR.status = status; + jqXHR.statusText = ( nativeStatusText || statusText ) + ""; + + // Success/Error + if ( isSuccess ) { + deferred.resolveWith( callbackContext, [ success, statusText, jqXHR ] ); + } else { + deferred.rejectWith( callbackContext, [ jqXHR, statusText, error ] ); + } + + // Status-dependent callbacks + jqXHR.statusCode( statusCode ); + statusCode = undefined; + + if ( fireGlobals ) { + globalEventContext.trigger( isSuccess ? "ajaxSuccess" : "ajaxError", + [ jqXHR, s, isSuccess ? success : error ] ); + } + + // Complete + completeDeferred.fireWith( callbackContext, [ jqXHR, statusText ] ); + + if ( fireGlobals ) { + globalEventContext.trigger( "ajaxComplete", [ jqXHR, s ] ); + + // Handle the global AJAX counter + if ( !( --jQuery.active ) ) { + jQuery.event.trigger( "ajaxStop" ); + } + } + } + + return jqXHR; + }, + + getJSON: function( url, data, callback ) { + return jQuery.get( url, data, callback, "json" ); + }, + + getScript: function( url, callback ) { + return jQuery.get( url, undefined, callback, "script" ); + } +} ); + +jQuery.each( [ "get", "post" ], function( i, method ) { + jQuery[ method ] = function( url, data, callback, type ) { + + // Shift arguments if data argument was omitted + if ( isFunction( data ) ) { + type = type || callback; + callback = data; + data = undefined; + } + + // The url can be an options object (which then must have .url) + return jQuery.ajax( jQuery.extend( { + url: url, + type: method, + dataType: type, + data: data, + success: callback + }, jQuery.isPlainObject( url ) && url ) ); + }; +} ); + + +jQuery._evalUrl = function( url, options ) { + return jQuery.ajax( { + url: url, + + // Make this explicit, since user can override this through ajaxSetup (#11264) + type: "GET", + dataType: "script", + cache: true, + async: false, + global: false, + + // Only evaluate the response if it is successful (gh-4126) + // dataFilter is not invoked for failure responses, so using it instead + // of the default converter is kludgy but it works. + converters: { + "text script": function() {} + }, + dataFilter: function( response ) { + jQuery.globalEval( response, options ); + } + } ); +}; + + +jQuery.fn.extend( { + wrapAll: function( html ) { + var wrap; + + if ( this[ 0 ] ) { + if ( isFunction( html ) ) { + html = html.call( this[ 0 ] ); + } + + // The elements to wrap the target around + wrap = jQuery( html, this[ 0 ].ownerDocument ).eq( 0 ).clone( true ); + + if ( this[ 0 ].parentNode ) { + wrap.insertBefore( this[ 0 ] ); + } + + wrap.map( function() { + var elem = this; + + while ( elem.firstElementChild ) { + elem = elem.firstElementChild; + } + + return elem; + } ).append( this ); + } + + return this; + }, + + wrapInner: function( html ) { + if ( isFunction( html ) ) { + return this.each( function( i ) { + jQuery( this ).wrapInner( html.call( this, i ) ); + } ); + } + + return this.each( function() { + var self = jQuery( this ), + contents = self.contents(); + + if ( contents.length ) { + contents.wrapAll( html ); + + } else { + self.append( html ); + } + } ); + }, + + wrap: function( html ) { + var htmlIsFunction = isFunction( html ); + + return this.each( function( i ) { + jQuery( this ).wrapAll( htmlIsFunction ? html.call( this, i ) : html ); + } ); + }, + + unwrap: function( selector ) { + this.parent( selector ).not( "body" ).each( function() { + jQuery( this ).replaceWith( this.childNodes ); + } ); + return this; + } +} ); + + +jQuery.expr.pseudos.hidden = function( elem ) { + return !jQuery.expr.pseudos.visible( elem ); +}; +jQuery.expr.pseudos.visible = function( elem ) { + return !!( elem.offsetWidth || elem.offsetHeight || elem.getClientRects().length ); +}; + + + + +jQuery.ajaxSettings.xhr = function() { + try { + return new window.XMLHttpRequest(); + } catch ( e ) {} +}; + +var xhrSuccessStatus = { + + // File protocol always yields status code 0, assume 200 + 0: 200, + + // Support: IE <=9 only + // #1450: sometimes IE returns 1223 when it should be 204 + 1223: 204 + }, + xhrSupported = jQuery.ajaxSettings.xhr(); + +support.cors = !!xhrSupported && ( "withCredentials" in xhrSupported ); +support.ajax = xhrSupported = !!xhrSupported; + +jQuery.ajaxTransport( function( options ) { + var callback, errorCallback; + + // Cross domain only allowed if supported through XMLHttpRequest + if ( support.cors || xhrSupported && !options.crossDomain ) { + return { + send: function( headers, complete ) { + var i, + xhr = options.xhr(); + + xhr.open( + options.type, + options.url, + options.async, + options.username, + options.password + ); + + // Apply custom fields if provided + if ( options.xhrFields ) { + for ( i in options.xhrFields ) { + xhr[ i ] = options.xhrFields[ i ]; + } + } + + // Override mime type if needed + if ( options.mimeType && xhr.overrideMimeType ) { + xhr.overrideMimeType( options.mimeType ); + } + + // X-Requested-With header + // For cross-domain requests, seeing as conditions for a preflight are + // akin to a jigsaw puzzle, we simply never set it to be sure. + // (it can always be set on a per-request basis or even using ajaxSetup) + // For same-domain requests, won't change header if already provided. + if ( !options.crossDomain && !headers[ "X-Requested-With" ] ) { + headers[ "X-Requested-With" ] = "XMLHttpRequest"; + } + + // Set headers + for ( i in headers ) { + xhr.setRequestHeader( i, headers[ i ] ); + } + + // Callback + callback = function( type ) { + return function() { + if ( callback ) { + callback = errorCallback = xhr.onload = + xhr.onerror = xhr.onabort = xhr.ontimeout = + xhr.onreadystatechange = null; + + if ( type === "abort" ) { + xhr.abort(); + } else if ( type === "error" ) { + + // Support: IE <=9 only + // On a manual native abort, IE9 throws + // errors on any property access that is not readyState + if ( typeof xhr.status !== "number" ) { + complete( 0, "error" ); + } else { + complete( + + // File: protocol always yields status 0; see #8605, #14207 + xhr.status, + xhr.statusText + ); + } + } else { + complete( + xhrSuccessStatus[ xhr.status ] || xhr.status, + xhr.statusText, + + // Support: IE <=9 only + // IE9 has no XHR2 but throws on binary (trac-11426) + // For XHR2 non-text, let the caller handle it (gh-2498) + ( xhr.responseType || "text" ) !== "text" || + typeof xhr.responseText !== "string" ? + { binary: xhr.response } : + { text: xhr.responseText }, + xhr.getAllResponseHeaders() + ); + } + } + }; + }; + + // Listen to events + xhr.onload = callback(); + errorCallback = xhr.onerror = xhr.ontimeout = callback( "error" ); + + // Support: IE 9 only + // Use onreadystatechange to replace onabort + // to handle uncaught aborts + if ( xhr.onabort !== undefined ) { + xhr.onabort = errorCallback; + } else { + xhr.onreadystatechange = function() { + + // Check readyState before timeout as it changes + if ( xhr.readyState === 4 ) { + + // Allow onerror to be called first, + // but that will not handle a native abort + // Also, save errorCallback to a variable + // as xhr.onerror cannot be accessed + window.setTimeout( function() { + if ( callback ) { + errorCallback(); + } + } ); + } + }; + } + + // Create the abort callback + callback = callback( "abort" ); + + try { + + // Do send the request (this may raise an exception) + xhr.send( options.hasContent && options.data || null ); + } catch ( e ) { + + // #14683: Only rethrow if this hasn't been notified as an error yet + if ( callback ) { + throw e; + } + } + }, + + abort: function() { + if ( callback ) { + callback(); + } + } + }; + } +} ); + + + + +// Prevent auto-execution of scripts when no explicit dataType was provided (See gh-2432) +jQuery.ajaxPrefilter( function( s ) { + if ( s.crossDomain ) { + s.contents.script = false; + } +} ); + +// Install script dataType +jQuery.ajaxSetup( { + accepts: { + script: "text/javascript, application/javascript, " + + "application/ecmascript, application/x-ecmascript" + }, + contents: { + script: /\b(?:java|ecma)script\b/ + }, + converters: { + "text script": function( text ) { + jQuery.globalEval( text ); + return text; + } + } +} ); + +// Handle cache's special case and crossDomain +jQuery.ajaxPrefilter( "script", function( s ) { + if ( s.cache === undefined ) { + s.cache = false; + } + if ( s.crossDomain ) { + s.type = "GET"; + } +} ); + +// Bind script tag hack transport +jQuery.ajaxTransport( "script", function( s ) { + + // This transport only deals with cross domain or forced-by-attrs requests + if ( s.crossDomain || s.scriptAttrs ) { + var script, callback; + return { + send: function( _, complete ) { + script = jQuery( " + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + +
+ +
+ + + + + + + + +
+ +
+ + + + + + +
+
+ +
+ + +
+
+ Contents +
+ +
+
+
+
+ +
+ +
+

Stationarity and Ergodicity

+
+

Overview

+

To be added.

+

Use the distribution flow figures from markov_prop.md, this time starting from +many different distributions.

+

Suggests ergodicity.

+

We will use the following imports

+
+
+
import numpy as np
+import scipy as sp
+import matplotlib.pyplot as plt
+import quantecon as qe
+from numba import njit
+
+from mpl_toolkits.mplot3d import Axes3D
+from mpl_toolkits.mplot3d.art3d import Poly3DCollection
+
+
+
+
+
+
+

Stationary Distributions

+

Let \((P_t)\) be a Markov semigroup on countable state space \(S\).

+

A distribution \(\psi \in \dD\) is called stationary for \((P_t)\) if

+
+\[ + \psi P_t = \psi + \text{ for all } t \geq 0 +\]
+

In many cases, it is easier to use the generator of the semigroup to identify +stationary distributions.

+

The next result makes this possible.

+

It is analogous to the idea that a point \(\bar x\) in \(\RR^d\) is stationary for +a vector ODE \(x'_t = F(x_t)\) when \(F(\bar x) = 0\).

+

It holds true under weaker conditions but the version stated here is easy to +prove and sufficient for most cases we consider.

+
+

Theorem 23

+
+

For a conservative intensity matrix \(Q\) on \(S\) and +associated Markov semigroup \((P_t)\), a distribution \(\psi\) on \(S\) is stationary +for \((P_t)\) if and only if \(\psi Q = 0\).

+
+
+

Proof. Suppose first that \(\psi \in \dD\) and \(\psi Q = 0\).

+

Since \(Q\) is conservative, \((P_t)\) is a UC semigroup with \(P_t = e^{tQ}\) for +all \(t\), and hence, for any \(t \geq 0\),

+
+\[ + \psi e^{tQ} = \psi + t \psi Q + t^2 \frac{\psi Q^2}{2!} + + \cdots +\]
+

From \(\psi Q = 0\) we get \(\psi Q^k = 0\) for all \(k \in \NN\), so the last +display yields \(\psi P_t = \psi\).

+

Hence \(\psi\) is stationary for \((P_t)\).

+

Now suppose that \(\psi\) is stationary for \((P_t)\) and set \(D_t := (1/t) (P_t - +I)\).

+

From the triangle inequality and the definition of the operator norm, for any given \(t\),

+
+\[ + \| \psi Q \| + \leq \| \psi (Q - D_t) \| + \| \psi D_t \| + \leq \| Q - D_t \| + \| \psi D_t \| +\]
+

Since \((P_t)\) is UC and \(Q\) is its generator, we have \(\| D_t - Q \| \to 0\) in +\(\lL(\ell_1)\) as \(t \to 0\) from the right.

+

Hence \(\| \psi Q \| \leq \liminf_{t \to 0} \| \psi D_t \|\).

+

As \(\psi\) is stationary, \(\psi D_t = 0\) for all \(t\).

+

Hence \(\psi Q = 0\), as was to be shown.

+
+
+
+

Irreducibility and Uniqueness

+

Let \((P_t)\) be a Markov semigroup on \(S\).

+

We say that state \(y\) is accessible from state \(x\) if +there exists a \(t \geq 0\) such that \(P_t(x, y) > 0\).

+

A Markov semigroup \((P_t)\) on \(S\) is called irreducible if \(y\) is +accessible from \(x\) for every \((x,y) \in S \times S\).

+

We seek a characterization of irreducibility of \((P_t)\) in terms of its +generator.

+

As a first step, we will say there is a \(Q\)-positive probability flow from \(x\) +to \(y\) if there exists a finite sequence \((z_i)_{i=1}^m\) in \(S\) starting at +\(x=z_1\) and ending at \(y=z_m\) such that \(Q(z_i, z_{i+1}) > 0\) for all \(i\).

+
+

Lemma 24

+
+

For distinct states \(x\) and \(y\), the following statements are equivalent:

+
    +

  1. The state \(y\) is accessible from \(x\) under \((P_t)\).

    2. +

  2. There is a positive probability flow from \(x\) to \(y\) under \(Q\).

    3. +
+
+

((prove only for the UC case?))

+
+

Proof. For distinct states \(x\) and \(y\), we have

+
+(54)\[ + P_t(x, y) = t Q(x,y) + \frac{t^2}{2!} Q^2(x, y) + \cdots +\]
+

If \(x\) is accessible from \(y\), then \(P_t(x, y) > 0\) for some \(t > 0\), and hence +\(Q^k(x,y) > 0\) for at least one \(k \in \NN\).

+

Writing out the matrix product as a sum, we now have

+
+(55)\[ + Q^k(x,y) = + \sum_{z_1} + \sum_{z_2} + \cdots + \sum_{z_{k-1}} + Q(x, z_1) Q(z_1, z_2) \cdots Q(z_{k-1}, y) + > 0 +\]
+

It follows that at least one element of the sum must be strictly positive, so +positive probability flow from \(x\) to \(y\) is established.

+

To prove the converse, suppose there is positive probability flow from \(x\) to +\(y\).

+

We can then take a finite sequence \((z_i)_{i=1}^m\) starting at \(x\) and ending +at \(y\) with \(Q(z_i, z_{i+1}) > 0\) for all \(i\).

+

We can and do assume that \((z_i)_{i=0}^m\) is the shortest such sequence +(otherwise just pick the shortest one).

+

Because any shorter sequence \((u_i)_{i=1}^k\) linking \(x\) and \(y\) has \(Q(u_i, u_{i+1}) = 0\) for some \(i\), (55) implies that \(Q^k(x,y) = 0\) for all \(k < m\).

+

Hence, by (54), we have \(P_t(x, y) = t^m Q^m(x, y) + o(t^m)\).

+

As \(Q^m(x, y) > 0\), we see that \(P_t(x, y) > 0\) for sufficiently small \(t\).

+
+

Irreducible implies aperiodic.

+
+

Theorem 25

+
+

For a ((UC)) Markov semigroup \((P_t)\), the following statements are +equivalent:

+
    +

  1. \((P_t)\) is irreducible.

    2. +

  2. \(P_t(x,y) > 0\) for all \(t > 0\) and all \((x,y) \in S \times S\).

    3. +
+
+
+

Proof. To be added.

+
+

Irreducible implies uniqueness.

+
+
+

Asymptotic Stabiltiy

+

To be added.

+

Include Foguel alternative?

+

Thm 6.2 of RR and MTK.

+
+
+ + + + +
+ +
+
+ + +
+ + A Probabilistic View + Bibliography + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file diff --git a/generators.html b/generators.html new file mode 100644 index 0000000..480785a --- /dev/null +++ b/generators.html @@ -0,0 +1,795 @@ + + + + + + + + Semigroups and Generators — Continuous Time Markov Chains + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + +
+ +
+ + + + + + + + +
+ +
+ + + + + + +
+
+ +
+ + +
+
+ Contents +
+ +
+
+
+
+ +
+ +
+

Semigroups and Generators

+
+

Overview

+

We have seen in previous lectures that every intensity matrix generates a +Markov semigroup.

+

We have also hinted that the pairing is one-to-one, in a sense to be made precise.

+

To clarify these ideas, we start in an +abstract setting, with an arbitrary initial value problem.

+

In this setting we introduce general operator semigroups and their generators.

+

Once this is done, we will be able to return to the Markov case and fully +clarify the connection between intensity matrices and Markov semigroups.

+

The material below is relatively technical, with most of the +complications driven by the fact that the state space can be infinite.

+

Such technicalities are hard to avoid, since so many interesting Markov chains +do have infinite state spaces.

+
    +

  • Our very first example – the Poisson process – has an infinite state space.

    • +

  • Another example is the study of queues, which often have no natural upper +bound.1

    • +
+

Readers are assumed to have some basic familiarity with Banach spaces.

+
+
+

Motivation

+

The general theory of continuous semigroups of operators is motivated by +the problem of solving linear ODEs in infinite dimensional spaces.2

+

More specifically, the challenge is to solve initial value problems such as

+
+(38)\[ + x'_t = A x_t, + \quad x_0 \text{ given} +\]
+

where

+
    +

  • \(x_t\) takes value in a Banach space at each time \(t\),

    • +

  • \(A\) is a linear operator and

    • +

  • the time derivative \(x'_t\) uses a definition appropriate for a Banach +space.

    • +
+

This problem is also called the “abstract Cauchy problem”.

+

Why do we need solve such problems?

+

One example comes from PDEs.

+

PDEs tell us how functions change over time, starting from an infinitesimal +description.

+

When \(x_t\) is a point in a function space, this fits into the framework of +(38).

+

Another example comes from Markov processes, where, as we have seen, the flow +of distributions over time can be represented as a linear ODE in distribution +space.

+

If the number of state is infinite, then the space of distributions is +infinite dimensional.

+

This is another version of (38), and we return to it after a discussion +of the general theory.

+

To give a high level view of the results below, the solution to the Cauchy +problem is represented as a trajectory \(t \mapsto U_t x_0\) from the initial +value \(x_0\) under a semigroup of maps \((U_t)\).

+

The operator \(A\) in (38) is called the “generator” of \((U_t)\) and is +its infinitesimal description.

+
+
+

Preliminaries

+

Throughout this lecture, \((\BB, \| \cdot \|)\) is a Banach space.

+
+

The Space of Linear Operators

+

You will recall that a linear operator on \(\BB\) is a map \(A\) from \(\BB\) to +itself satisfying

+
+\[ + A(\alpha g + \beta h) = \alpha A g + \beta A h, + \quad + \forall \, g, h \in \BB, \;\; \alpha, \beta \in \RR +\]
+

The operator \(A\) is called bounded if

+
+(39)\[ + \| A \| := \sup_{g \in \BB, \, \| g \| \leq 1} \| A g \| < \infty +\]
+

This is the usual definition of a bounded linear operator on a normed linear space.

+

The set of all bounded linear operators on \(\BB\) is denoted by \(\linop\) and is +itself a Banach space.

+

Sums and scalar products of elements of \(\linop\) are defined in the usual way, +so that, for \(\alpha \in \RR\), \(A, B \in \linop\) and \(g \in \BB\), we have

+
+\[ + (A + B) g = Ag + Bg, + \quad (\alpha A) g = \alpha (A g) +\]
+

and so on.

+

We write \(A B\) to indicate composition of the operators \(A, B \in \linop\).

+

The value defined in (39) is called the operator norm of \(A\) and, +as suggested by the notation, is a norm on \(\linop\).

+

In addition to being a norm, it enjoys the submultiplicative property \(\| AB +\| \leq \| A \| \| B\|\) for all \(A, B \in \linop\).

+

Let \(I\) be the identity in \(\linop\), satisfying \(Ig = g\) for all \(g \in \BB\).

+

(In fact \(\linop\) is a unital Banach algebra when multiplication is identified with operator composition and \(I\) is adopted as the unit.)

+
+
+

The Exponential Function

+

Given \(A \in \linop\), the exponential of \(A\) is the element of +\(\linop\) defined as

+
+(40)\[ + e^A + = \sum_{k \geq 0} \frac{A^k}{k!} + = I + A + \frac{A^2}{2!} + \cdots +\]
+

This is the same as the definition for the matrix exponential.The exponential function arises naturally as the solution to ODEs in Banach +space, one example of which (as we shall see) is distribution flows +associated with continuous time Markov chains.

+

The exponential map has the following properties:

+
    +

  • For each \(A \in \linop\), the operator \(e^A\) is a well defined element of \(\linop\) with \(\| e^A \| \leq e^{\| A \|}\).3

    • +

  • \(e^0 = I\), where \(0\) is the zero element of \(\linop\).

    • +

  • If \(A, B \in \linop\) and \(AB = BA\), then \(e^{A + B} = e^A e^B\)

    • +

  • If \(A \in \linop\), then \(e^A\) is invertible and \((e^A)^{-1} = e^{-A}\).

    • +
+

The last fact is easily checked from the previous ones.

+
+
+

Operator Calculus

+

Consider a function

+
+\[ + \RR_+ \ni t \mapsto U_t \in \linop +\]
+

which we can think of as a time path in \(\linop\), such as a flow of Markov +operators.

+

We say that this function is differentiable at \(\tau \in \RR_+\) if there exists +an element \(T\) of \(\linop\) such that

+
+(41)\[ + \frac{U_{\tau+h} - U_\tau}{h} \to T + \; \text{ as } h \to 0 +\]
+

In this case, \(T\) is called the derivative of the function \(t \mapsto U_t\) at \(\tau\) and we write

+
+\[ + T = U'_\tau + \; \text{ or } \; + T = \frac{d}{dt} U_t \, \Big|_{t=\tau} +\]
+

(Convergence of operators is in operator norm. If \(\tau = 0\), then the limit +\(h \to 0\) in (41) is the right limit.)

+
+

Example 12

+
+

If \(U_t = t V\) for some fixed \(V \in \linop\), then it is easy to +see that \(V\) is the derivative of \(t \mapsto U_t\) at every \(t \in \RR_+\).

+
+
+

Example 13

+
+

In our discussion of the Kolmogorov forward equation +when \(S\) is finite, we introduced the derivative of a map \(t +\mapsto P_t\), where each \(P_t\) is a matrix on \(S\).

+

The derivative was defined by differentiating \(P_t\) element-by-element.

+

This coincides with the operator-theoretic definition in (41) when \(S\) +is finite, because then the space \(\lL(\ell_1)\) is finite dimensional, and +hence pointwise and norm convergence coincide.

+
+

Analogous to the matrix and scalar cases, we have the following result:

+
+

Lemma 14 (Differentiability of the Exponential Curve)

+
+

For all \(A \in \linop\), the exponential curve \(t \mapsto e^{tA}\) is everywhere differentiable and

+
+(42)\[ + \frac{d}{dt} e^{tA} = e^{tA} A = A e^{tA} +\]
+
+

The proof is a (solved) exercise (see below).

+
+
+
+

Semigroups and Generators

+

For continuous time Markov chains where the state space \(S\) is finite, we +saw that Markov semigroups often take the form \(P_t = e^{tQ}\) for some +intensity matrix \(Q\).

+

This is ideal because the entire semigroup is characterized in a simple way by +its infinitesimal description \(Q\).

+

It turns out that, when \(S\) is finite, this is always true: if \((P_t)\) is a +Markov semigroup, then there exists an intensity matrix \(Q\) satisfying \(P_t = +e^{tQ}\) for all \(t\).

+

Moreover, this statement is again true when \(S\) is infinite, provided that +some restrictions are placed on the semigroup.

+

Our aim is to make these statements precise, starting in an abstract setting +and then specializing.

+
+

Operator Semigroups

+

Let \(U_t\) be an element of \(\linop\) for all \(t \in \RR_+\)

+

We say that \((U_t)\) is an evolution semigroup on \(\linop\) if \(U_0 = I\) and +\(U_{s + t} = U_s U_t\) for all \(s, t \geq 0\).

+

The idea is that \((U_t)\) generates a path in \(\BB\) from any starting point \(g \in \BB\), so that \(U_t g\) is interpreted as the location of the state after \(t\) units of time.

+

An evolution semigroup \((U_t)\) is called

+
    +

  • a \(C_0\) semigroup on \(\BB\) if, for each \(g \in \BB\), the map \(t \mapsto U_t g\) from \(\RR_+\) to \(\BB\) is continuous, and

    • +

  • a uniformly continuous semigroup on \(\BB\) if the map \(t \mapsto U_t\) from \(\RR_+\) to \(\linop\) is continuous.

    • +
+

In what follows we abbreviate “uniformly continuous” to UC.4

+
+

Example 15 (Exponential curves are UC semigroups)

+
+

If \(U_t = e^{tA}\) for \(t \in \RR_+\) and \(A \in \linop\), then \((U_t)\) +is a uniformly continuous semigroup on \(\BB\).

+
+

The claim that \((U_t)\) is an evolution semigroup follows directly from the +properties of the exponential function given above.

+

Uniform continuity can be established using arguments similar to those in the +proof of differentiability in Lemma 14.

+

Since norm convergence on \(\linop\) implies pointwise convergence, every +uniformly continuous semigroup is a \(C_0\) semigroup.

+

The reverse is certainly not true — there are many important \(C_0\) semigroups that fail to be uniformly continuous.

+

In fact semigroups associated with PDEs, diffusions and other Markov processes +on continuous state spaces are typically \(C_0\) but not uniformly continuous.

+

There are also important examples of Markov semigroups on infinite +discrete state spaces that fail to be uniformly continuous.

+

However, we will soon see that, for most continuous time Markov chains used in +applications, the semigroups are uniformly continuous.

+
+
+

Generators

+

Consider a continuous time Markov chain on a finite state space with intensity +matrix \(Q\).

+

The Markov semigroup \((P_t)\) is fully specified by this infinitesimal +description \(Q\), in the sense that

+
    +

  • \(P_t = e^{tQ}\) for all \(t \geq 0\) and (equivalently)

    • +

  • the forward and backward equations hold: \(P_t' = P_t Q = Q P_t\).

    • +
+

Since \(P_0 = I\), the matrix \(Q\) can be recovered from the semigroup via

+
+\[ + Q = P'_0 = \lim_{h \downarrow 0} \frac{P_h - I}{h} +\]
+

In the more abstract setting of \(C_0\) semigroups, we say that \(Q\) is the +“generator” of the semigroup \(P_t\).

+

More generally, given a \(C_0\) semigroup \((U_t)\), we say that a linear +operator \(A\) from \(\BB\) to itself is the generator of \((U_t)\) if

+
+(43)\[ + A g = \lim_{h \downarrow 0} \frac{U_h g - g}{h} +\]
+

for all \(g \in \BB\) such that the limit exists.

+

The set of points where the limit exists (the domain of the generator) is +denoted by \(D(A)\).

+

At this point we would like to write (43) as \(A = U'_0\), or express +\(U_t\) as \(e^{tA}\), analogous to the Markov case.

+

There are problems, however.

+

One problem is that the limit in (43) can fail to exist for some \(g +\in \BB\).

+

Indeed, why should the limit exist, given that \(C_0\) semigroups are not +required to be differentiable?

+

The other problem is that, even though the limit exists, the linear operator +\(A\) is not bounded (i.e., not an element of \(\linop\)), so +a statement like \(U_t = e^{tA}\) is problematic.

+

It turns out that, despite these issues, the theory of \(C_0\) semigroups is +powerful, and, with some work, the technical issues can be +circumvented.5

+

Even better, for the applications we wish to consider, we can focus on UC +semigroups, where these problems do not arise.

+

The next section gives details.

+
+
+

A Characterization of Uniformly Continuous Semigroups

+

We saw in Example 15 that exponential curves are an example +of a UC semigroup.

+

The next theorem tells us that there are no other examples.

+
+

Theorem 16 (UC Semigroups are Exponential Curves)

+
+

If \((U_t)\) is a UC semigroup on \(\BB\), then there exists an \(A \in \linop\) +such that \(U_t = e^{tA}\) for all \(t \geq 0\). Moreover,

+
    +

  • \(U_t\) is differentiable at every \(t \geq 0\),

    • +

  • \(A\) is the generator of \((U_t)\) and

    • +

  • \(U_t' = A U_t = U_t A\) for all \(t \geq 0\).

    • +
+
+

The last three claims in Theorem 16 follow directly from the +first claim.

+

The statement \(U_t' = A U_t = U_t A\) is a +generalization of the Kolmogorov forward and backward equations.

+

While slightly more complicated in the Banach setting, the proof of the first +claim (existence of an exponential representation) is a direct extension of +the fact that any continuous function \(f\) from \(\RR\) to itself +satisfying

+
    +

  • \(f(s)f(t) = f(s+t)\) for all \(s,t \geq 0\) and

    • +

  • \(f(0) = 1\)

    • +
+

also satisfies \(f(t) = e^{ta}\) for some \(a \in \RR\).

+

We proved something quite similar in Theorem 1, on +the memoryless property of the exponential function.

+

For more discussion of the scalar case, see, for example, +[SK11].

+

For a full proof of the first claim in Theorem 16, in the setting of +a Banach algebra, see, for +example, Chapter 7 of [Bob05].

+
+
+
+

Exercises

+
+

Exercise 1

+

Prove that (42) holds for all \(A \in \linop\).

+
+
+

Exercise 2

+

In many texts, a \(C_0\) semigroup is defined as an evolution semigroup \((U_t)\) +such that

+
+(44)\[ + U_t g \to g \text{ as } t \to 0 \text{ for any } g \in \BB +\]
+

Our aim is to show that (44) implies continuity at every point \(t\), as +in the definition we used above.

+

The Banach–Steinhaus Theorem can be used to show that, for an evolution semigroup \((U_t)\) satisfying (44), there exist finite constants \(\omega\) and \(M\) such that

+
+(45)\[ + \| U_t \| \leq e^{t\omega} M + \quad \text{for all } \; t \geq 0 +\]
+

Using this and (44), show that, for any \(g \in \BB\), the map \(t \mapsto +U_t g\) is continuous at all \(t\).

+
+
+

Exercise 3

+

Following on from the previous exercise, +a UC semigroup is often defined as an evolution semigroup \((U_t)\) +such that

+
+(46)\[ + \| U_t - I \| \to 0 \text{ as } t \to 0 +\]
+

Show that (46) implies norm continuity at every point \(t\), as +in the definition we used above.

+

In particular, show that, for any \(t_n \to t\), we have +\(\| U_{t_n} - U_t \| \to 0\) as \(n \to \infty\).

+
+
+
+

Solutions

+
+

Solution to Exercise 1

+

To show the first equality, fix \(t \in \RR_+\), take \(h > 0\) and observe that

+
+\[ + e^{(t+h)A} - e^{tA} - e^{tA} A + = e^{tA} (e^{hA} - I - A) +\]
+

Since the norm on \(\linop\) is submultiplicative, it suffices to show that +\(\| e^{hA} - I - A \| = o(h)\) as \(h \to 0\).

+

Using the definition of the exponential, this is easily verified, +completing the proof of the first equality in (42).

+

The proof of the second equality is similar.

+
+
+

Solution to Exercise 2

+

Let \((U_t)\) be an evolution semigroup satisfying (44) and let +\(\omega\) and \(M\) be as in (45).

+

Pick any \(g \in \BB\), \(t > 0\) and \(h_n \downarrow 0\).

+

On one hand, \(U_{t+ h_n} g = U_{h_n} U_t g \to U_t g\) by (44).

+

On the other hand, from (45) and the definition of the operator norm,

+
+\[ + \| U_{t-h_n} g - U_t g\| + = \| U_{t-h_n} ( g - U_{h_n} g) \| + \leq e^{(t-h_n)\omega} M \| g - U_{h_n} g\| + \to 0 +\]
+

as \(n \to \infty\). This completes the proof.

+
+
+

Solution to Exercise 3

+

The solution is similar to that of the previous exercise.

+

Let \((U_t)\) be an evolution semigroup satisfying (46), +fix \(t > 0\) and take \((h_n)\) to be a scalar sequence satisfying \(h_n \downarrow 0\).

+

On one hand, \(U_{t+ h_n} = U_{h_n} U_t \to U_t \) by (46).

+

On the other hand, from the submultiplicative property of the operator norm +and (45),

+
+\[ + \| U_{t-h_n} - U_t \| + = \| U_{t-h_n} ( I - U_{h_n}) \| + \leq e^{(t-h_n)\omega} M \| I - U_{h_n} \| +\]
+

This converges to 0 as \(n \to \infty\), completing our proof.

+
+
+
1
+

In fact a major concern with queues is that their length does not explode. This issue cannot be properly explored unless the state space is allowed to be infinite.

+
+
2
+

An excellent introduction to operator semigroups, combined with +applications to PDEs and Markov processes, can be found in [applebaum2019semigroups].

+
+
3
+

Convergence of the sum in (40) follows from boundedness of \(A\) and the fact that \(\linop\) is a Banach space.

+
+
4
+

Be careful: the definition of a UC semigroup requires that +\(t \mapsto U_t\) is continuous as a map into \(\linop\), rather than uniformly +continuous. The UC terminology comes about because, for a UC semigroup, we +have, by definition of the operator norm, +\(\sup_{\| g \| \leq 1} \| U_s g - U_t g \| \to 0\) when \(s \to t\).

+
+
5
+

An excellent treatment of the general theory of \(C_0\) semigroups, can be found in [Bob05].

+
+
+
+
+
+ + + + +
+ +
+
+ + +
+ + The Kolmogorov Forward Equation + UC Markov Semigroups + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file diff --git a/genindex.html b/genindex.html new file mode 100644 index 0000000..28743b0 --- /dev/null +++ b/genindex.html @@ -0,0 +1,230 @@ + + + + + + + + + Index — Continuous Time Markov Chains + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + + + +
+ + +
+ +
+
+
+
+ +
+ + +

Index

+ +
+ +
+ + +
+ +
+
+ + +
+ + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file diff --git a/index.html b/index.html new file mode 100644 index 0000000..de49afb --- /dev/null +++ b/index.html @@ -0,0 +1,2 @@ + + diff --git a/intro.html b/intro.html new file mode 100644 index 0000000..bf514f9 --- /dev/null +++ b/intro.html @@ -0,0 +1,266 @@ + + + + + + + + Continuous Time Markov Chains — Continuous Time Markov Chains + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + +
+ +
+ + + + + + + + +
+ + +
+
+ Contents +
+ +
+
+
+
+ +
+ +
+

Continuous Time Markov Chains

+

Introductory text to be added.

+

Uses — list applications.

+
+
+
+ + + + +
+ +
+
+ + +
+ + Memoryless Distributions + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file diff --git a/kolmogorov_bwd.html b/kolmogorov_bwd.html new file mode 100644 index 0000000..32a9a5a --- /dev/null +++ b/kolmogorov_bwd.html @@ -0,0 +1,954 @@ + + + + + + + + Kolmogorov Backward Equation — Continuous Time Markov Chains + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + +
+ +
+ + + + + + + + +
+ +
+ + + + + + +
+
+ +
+ + +
+
+ Contents +
+ +
+
+
+
+ +
+ +
+

Kolmogorov Backward Equation

+
+

Overview

+

As models become more complex, deriving analytical representations of the +Markov semigroup \((P_t)\) becomes harder.

+

This is analogous to the idea that solutions to continuous time models often +lack analytical solutions.

+

For example, when studying deterministic paths in continuous time, +infinitesimal descriptions (ODEs and PDEs) are often more intuitive and easier +to write down than the associated solutions.

+

(This is one of the shining insights of mathematics, beginning with the work +of great scientists such as Isaac Newton.)

+

We will see in this lecture that the same is true for continuous time Markov +chains.

+

To help us focus on intuition in this lecture, rather than technicalities, the state space is assumed to be finite, with \(|S|=n\).

+

Later we will investigate the case where \(|S| = \infty\).

+

We will use the following imports

+
+
+
import numpy as np
+import scipy as sp
+import matplotlib.pyplot as plt
+import quantecon as qe
+from numba import njit
+
+from scipy.linalg import expm
+from scipy.stats import binom
+
+
+
+
+
+
+

State Dependent Jump Intensities

+

As we have seen, continuous time Markov chains jump between states, and hence can +have the form

+
+\[ + X_t = \sum_{k \geq 0} Y_k \mathbb 1\{J_k \leq t < J_{k+1}\} + \qquad (t \geq 0) +\]
+

where \((J_k)\) are jump times and \((Y_k)\) are the states at each jump.

+

(We are assuming that \(J_k \to \infty\) with probability one, so that \(X_t\) is well +defined for all \(t \geq 0\), but this is always true for when holding times are exponential and the state space is finite.)

+

In the previous lecture,

+
    +

  • the sequence \((Y_k)\) was drawn from a Markov matrix \(K\) and called the embedded jump chain, while

    • +

  • the holding times \(W_k := J_k - J_{k-1}\) were IID and Exp\((\lambda)\) for some +constant jump intensity \(\lambda\).

    • +
+

In this lecture, we will generalize by allowing the jump intensity to vary +with the state.

+

This difference sounds minor but in fact it will allow us to reach full generality +in our description of continuous time Markov chains, as +clarified below.

+
+

Motivation

+

As a motivating example, recall the inventory model, +where we assumed that the wait time for the next customer was equal +to the wait time for new inventory.

+

This assumption was made purely for convenience and seems unlikely to hold true.

+

When we relax it, the jump intensities depend on the state.

+
+
+

Embedded Jump Chain Algorithm

+

We start with three primitives

+
    +

  1. An initial condition \(\psi\),

    2. +

  2. a Markov matrix \(K\) on \(S\) satisfying \(K(x, x) = 0\) for all \(x \in S\) +and

    3. +

  3. a function \(\lambda\) mapping \(S\) to \((0, \infty)\).

    4. +
+

The process \((X_t)\)

+
    +

  • starts at state \(x\), draw from \(\psi\),

    • +

  • waits there for an exponential time \(W\) with rate \(\lambda(x)\) and then

    • +

  • updates to a new state \(y\) drawn from \(K(x, \cdot)\).

    • +
+

Now we take \(y\) as the new state for the process and repeat.

+

Here is the same algorithm written more explicitly:

+
+

Algorithm 6 (Embedded Jump Chain Algorithm)

+
+

Inputs \(\psi \in \dD\), rate function \(\lambda\), Markov matrix \(K\)

+

Outputs Markov chain \((X_t)\)

+
    +

  1. draw \(Y_0\) from \(\psi\), set \(J_0 = 0\) and \(k=1\).

    2. +

  2. Draw \(W_k\) independently from Exp\((\lambda(Y_{k-1}))\).

    3. +

  3. Set \(J_k = J_{k-1} + W_k\).

    4. +

  4. Set \(X_t = Y_{k-1}\) for \(t\) in \([J_{k-1}, J_k)\).

    5. +

  5. Draw \(Y_k\) from \(K(Y_{k-1}, \cdot)\).

    6. +

  6. Set \(k = k+1\) and go to step 2.

    7. +
+
+

The sequence \((W_k)\) is drawn as an IID sequence and \((W_k)\) and \((Y_k)\) are +drawn independently.

+

The restriction \(K(x,x) = 0\) for all \(x\) implies that \((X_t)\) actually jumps at each jump time.

+
+
+
+

Computing the Semigroup

+

For the jump process \((X_t)\) with time varying intensities described in the +jump chain algorithm, calculating the Markov semigroup is not a trivial exercise.

+

The approach we adopt is

+
    +

  1. use probabilistic reasoning to obtain an integral equation that the +semigroup must satisfy.

    2. +

  2. Convert the integral equation into a differential equation that is easier +to work with.

    3. +

  3. Solve this differential equation to obtain the Markov semigroup \((P_t)\).

    4. +
+

The differential equation in question has a special name: the Kolmogorov backward equation.

+
+

An Integral Equation

+

Here is the first step in the sequence listed above.

+
+

Lemma 7 (An Integral Equation)

+
+

The semigroup \((P_t)\) of the jump chain with rate function \(\lambda\) and Markov matrix \(K\) obeys the integral equation

+
+(21)\[ + P_t(x, y) = e^{-t \lambda(x)} I(x, y) + + \lambda(x) + \int_0^t (K P_{t-\tau})(x, y) e^{- \tau \lambda(x)} d \tau +\]
+

for all \(t \geq 0\) and \(x, y\) in \(S\).

+
+

Here \((P_t)\) is the Markov semigroup of \((X_t)\), the process constructed via +Algorithm 6, while \(K P_{t-\tau}\) is the matrix product of \(K\) and +\(P_{t-\tau}\).

+
+

Proof. Conditioning implicitly on \(X_0 = x\), the semigroup \((P_t)\) must satisfy

+
+(22)\[ + P_t(x, y) + = \PP\{X_t = y\} + = \PP\{X_t = y, \; J_1 > t \} + + \PP\{X_t = y, \; J_1 \leq t \} +\]
+

Regarding the first term on the right hand side of (22), we have

+
+(23)\[ + \PP\{X_t = y, \; J_1 > t \} + = I(x, y) P\{J_1 > t \} + = I(x, y) e^{- t \lambda(x)} +\]
+

where \(I(x, y) = \mathbb 1\{x = y\}\).

+

For the second term on the right hand side of (22), we have

+
+\[ + \PP\{X_t = y, \; J_1 > t \} + = \EE + \left[ + \mathbb 1\{J_1 > t\} \PP\{X_t = y \,|\, W_1, Y_1\} + \right] + = \EE + \left[ + \mathbb 1\{J_1 > t\} P_{t - J_1} (Y_1, y) + \right] +\]
+

Evaluating the expectation and using the independence of \(J_1\) and \(Y_1\), this becomes

+
+\[\begin{split} +\begin{aligned} + \PP\{X_t = y, \; J_1 > t \} + & = \int_0^\infty + \mathbb 1\{\tau > t\} + \sum_z K(x, z) P_{t - \tau} (z, y) \lambda(x) e^{-\tau \lambda(x)} + d \tau + \\ + & = \lambda(x) + \int_0^t + \sum_z K(x, z) P_{t - \tau} (z, y) e^{-\tau \lambda(x)} + d \tau +\end{aligned} +\end{split}\]
+

Combining this result with (22) and (23) gives +(21).

+
+
+
+

Kolmogorov’s Differential Equation

+

We have now confirmed that the semigroup \((P_t)\) associated with the jump +chain process \((X_t)\) satisfies (21).

+

Equation (21) is important but we can simplify it further without +losing information by taking the time derivative.

+

This leads to our main result for the lecture

+
+

Theorem 8 (Kolmogorov Backward Equation)

+
+

The semigroup \((P_t)\) of the jump chain with rate function \(\lambda\) and Markov matrix \(K\) satisfies the Kolmogorov backward equation

+
+(24)\[ + P'_t = Q P_t + \quad \text{where } \; + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +\]
+
+

The derivative on the left hand side of (24) is taken element by +element, with respect to \(t\), so that

+
+\[ + P'_t(x, y) = \left( \frac{d}{dt} P_t(x, y) \right) + \qquad ((x, y) \in S \times S) +\]
+

The proof that differentiating (21) yields (24) is an +important exercise (see below).

+
+
+

Exponential Solution

+

The Kolmogorov backward equation is a matrix-valued differential equation.

+

Recall that, for a scalar differential equation \(y'_t = a y_t\) with constant +\(a\) and initial condition \(y_0\), the solution is \(y_t = e^{ta} y_0\).

+

This, along with \(P_0 = I\), encourages us to guess that the solution to +Kolmogorov’s backward equation (24) is

+
+(25)\[ + P_t = e^{t Q} +\]
+

where the right hand side is the matrix exponential, with definition

+
+(26)\[ + e^{tQ} + = \sum_{k \geq 0} \frac{1}{k!} (tQ)^k + = I + tQ + \frac{t^2}{2!} Q^2 + \cdots +\]
+

Working element by element, it is not difficult to confirm that +the derivative of the exponential function \(t \mapsto e^{tQ}\) is

+
+(27)\[ + \frac{d}{dt} e^{t Q} = Q e^{t Q} = e^{t Q} Q +\]
+

Hence, differentiating (25) gives \(P'_t = Q e^{t Q} = Q P_t\), which +convinces us that the exponential solution satisfies (24).

+

Notice that our solution

+
+(28)\[ + P_t = e^{t Q} + \quad \text{where } \; + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +\]
+

for the semigroup of the jump process \((X_t)\) associated with the jump matrix +\(K\) and the jump intensity function \(\lambda \colon S \to (0, \infty)\) is +consistent with our earlier result.

+

In particular, we showed that, for the model with +constant jump intensity \(\lambda\), we have \(P_t = e^{t \lambda (K - I)}\).

+

This is obviously a special case of (28).

+
+
+
+

Properties of the Solution

+

Let’s investigate further the properties of the exponential solution.

+
+

Checking the Transition Semigroup Properties

+

While we have confirmed that \(P_t = e^{t Q}\) solves the Kolmogorov backward +equation, we still need to check that this solution is a Markov semigroup.

+
+

Lemma 9 (From Jump Chain to Semigroup)

+
+

Let \(\lambda\) map \(S\) to \(\RR_+\) and let \(K\) be a Markov matrix on \(S\). +If \(P_t = e^{t Q}\) for all \(t \geq 0\), where +\(Q(x, y) = \lambda(x) (K(x, y) - I(x, y))\), then \((P_t)\) is a Markov +semigroup on \(S\).

+
+
+

Proof. Observe first that \(Q\) has zero row sums, since

+
+\[ + \sum_y Q(x, y) + = \lambda(x) \sum_y (K(x, y) - I(x, y)) + = 0 +\]
+

As a small exercise, you can check that the following is true

+
+(29)\[ + Q \text{ has zero row sums } + \iff + Q^k 1 = 0 \text{ for all } k \geq 1 +\]
+

This implies that \(Q^k 1 = 0\) for all \(k\) and, as a result, for any \(t \geq +0\),

+
+\[ + P_t 1 + = e^{tQ} 1 + = I1 + tQ1 + \frac{t^2}{2!} Q^2 1 + \cdots + = I1 = 1 +\]
+

In other words, each \(P_t\) has unit row sums.

+

Next we check positivity of all elements of \(P_t\) (which can easily fail for +matrix exponentials).

+

To this end, adopting an argument from [Str13], we +set \(m := \max_x \lambda(x)\) and \(\hat P := I + Q / m\).

+

It is not difficult to check that \(\hat P\) is a Markov matrix and \(Q = m( \hat +P - I)\).

+

Recalling that, for matrix exponentials, \(e^{A+B} = e^A e^B\) whenever \(AB = +BA\), we have

+
+\[ + e^{tQ} + = e^{tm (\hat P - I)} + = e^{-tm I} e^{tm \hat P} + = e^{-tm} + \left( + I + tm \hat P + \frac{(tm)^2}{2!} \hat P^2 + \cdots + \right) +\]
+

It is clear from this representation that all entries of \(e^{tQ}\) are +nonnegative.

+

Finally, we need to check the continuity condition \(P_t(x, y) \to I(x,y)\) as +\(t \to 0\), which is also part of the definition of a Markov semigroup. +This is immediate, in the present case, because the exponential function is +continuous, and hence \(P_t = e^{tQ} \to e^0 = I\).

+
+

We can now be reassured that our solution to the Kolmogorov backward equation +is indeed a Markov semigroup.

+
+
+

Uniqueness

+

Might there be another, entirely different Markov semigroup that also +satisfies the Kolmogorov backward equation?

+

The answer is no: linear ODEs in finite dimensional vector space with +constant coefficients and fixed initial conditions (in this case \(P_0 = I\)) +have unique solutions.

+

In fact it’s not hard to supply a proof — see the exercises.

+
+
+
+

Application: The Inventory Model

+

Let us look at a modified version of the inventory model where jump +intensities depend on the state.

+

In particular, the wait time for new inventory will now be exponential at rate +\(\gamma\).

+

The arrival rate for customers will still be denoted by \(\lambda\) and allowed +to differ from \(\gamma\).

+

For parameters we take

+
+
+
α = 0.6
+λ = 0.5
+γ = 0.1
+b = 10
+
+
+
+
+

Our plan is to investigate the distribution \(\psi_T\) of \(X_T\) at \(T=30\).

+

We will do this by simulating many independent draws of \(X_T\) and +histogramming them.

+

(In the exercises you are asked to calculate \(\psi_T\) a different way, via +(28).)

+
+
+
@njit
+def draw_X(T, X_0, max_iter=5000):
+    """
+    Generate one draw of X_T given X_0.
+    """
+
+    J, Y = 0, X_0
+    m = 0
+
+    while m < max_iter:
+        s = 1/γ if Y == 0 else 1/λ
+        W = np.random.exponential(scale=s)  # W ~ E(λ)
+        J += W
+        if J >= T:
+            return Y
+        # Otherwise update Y
+        if Y == 0:
+            Y = b
+        else:
+            U = np.random.geometric(α)
+            Y = Y - min(Y, U)
+        m += 1
+
+
+@njit
+def independent_draws(T=10, num_draws=100):
+    "Generate a vector of independent draws of X_T."
+
+    draws = np.empty(num_draws, dtype=np.int64)
+
+    for i in range(num_draws):
+        X_0 = np.random.binomial(b+1, 0.25)
+        draws[i] = draw_X(T, X_0)
+
+    return draws
+
+
+
+
+
+
+
T = 30
+n = b + 1 
+draws = independent_draws(T, num_draws=100_000)
+fig, ax = plt.subplots()
+
+ax.bar(range(n), [np.mean(draws == i) for i in range(n)], width=0.8, alpha=0.6)
+ax.set(xlabel="inventory")
+
+plt.show()
+
+
+
+
+_images/kolmogorov_bwd_6_0.png +
+
+

If you experiment with the code above, you will see that the large amount of +mass on zero is due to the low arrival rate \(\gamma\) for inventory.

+
+
+

Exercises

+
+

Exercise 1

+

In the discussion above, we generated an approximation of \(\psi_T\) when +\(T=30\), the initial condition is Binomial\((n, 0.25)\) and parameters +are set to

+
+
+
α = 0.6
+λ = 0.5
+γ = 0.1
+b = 10
+
+
+
+
+

The calculation was done by simulating independent draws and histogramming.

+

Try to generate the same figure using (28) instead, modifying code from +our lecture on the Markov property.

+
+
+

Exercise 2

+

Prove that differentiating (21) at each \((x, y)\) yields (24).

+
+
+

Exercise 3

+

We claimed above that the solution \(P_t = e^{t Q}\) is the unique +Markov semigroup satisfying the backward equation \(P'_t = Q P_t\).

+

Try to supply a proof.

+

(This is not an easy exercise but worth thinking about in any case.)

+
+
+
+

Solutions

+
+

Solution to Exercise 1

+

Here is one solution:

+
+
+
states = np.arange(n)
+I = np.identity(n)
+
+# Embedded jump chain matrix
+K = np.zeros((n, n))
+K[0, -1] = 1
+for i in range(1, n):
+    for j in range(0, i):
+        if j == 0:
+            K[i, j] = (1 - α)**(i-1)
+        else:
+            K[i, j] = α * (1 - α)**(i-j-1)
+
+# Jump intensities as a function of the state
+r = np.ones(n) * λ
+r[0] = γ
+
+# Q matrix
+Q = np.empty_like(K)
+for i in range(n):
+    for j in range(n):
+        Q[i, j] = r[i] * (K[i, j] - I[i, j])
+
+def P_t(ψ, t):
+    return ψ @ expm(t * Q)
+
+ψ_0 = binom.pmf(states, n, 0.25)
+ψ_T = P_t(ψ_0, T)
+
+fig, ax = plt.subplots()
+
+ax.bar(range(n), ψ_T, width=0.8, alpha=0.6)
+ax.set(xlabel="inventory")
+
+plt.show()
+
+
+
+
+_images/kolmogorov_bwd_10_0.png +
+
+
+
+

Solution to Exercise 2

+

One can easily verify that, when \(f\) is a differentiable function and \(\alpha > +0\), we have

+
+(30)\[ + g(t) = e^{- t \alpha} f(t) + \quad \implies \quad + g'(t) = e^{- t \alpha} f'(t) - \alpha g(t) +\]
+

Note also that, with the change of variable \(s = t - \tau\), we can rewrite +(21) as

+
+(31)\[ + P_t(x, y) = + e^{-t \lambda(x)} + \left\{ + I(x, y) + + \lambda(x) + \int_0^t (K P_s)(x, y) e^{s \lambda(x)} d s + \right\} +\]
+

Applying (30) yields

+
+\[ + P'_t(x, y) + = e^{-t \lambda(x)} + \left\{ + \lambda(x) + (K P_t)(x, y) e^{t \lambda(x)} + \right\} + - \lambda(x) P_t(x, y) +\]
+

After minor rearrangements this becomes

+
+\[ + P'_t(x, y) + = \lambda(x) [ (K - I) P_t](x, y) +\]
+

which is identical to (24).

+
+
+

Solution to Exercise 3

+

Here is one proof of uniqueness.

+

Suppose that \((\hat P_t)\) is another Markov semigroup satisfying +\(P'_t = Q P_t\).

+

Fix \(t > 0\) and let \(V_s\) be defined by \(V_s = P_s \hat P_{t-s}\) for all \(s +\geq 0\).

+

Note that \(V_0 = \hat P_t\) and \(V_t = P_t\).

+

Note also that \(s \mapsto V_s\) is differentiable, with derivative

+
+\[ + V'_s + = P'_s \hat P_{t-s} - P_s \hat P'_{t-s} + = P_s Q \hat P_{t-s} - P_s Q \hat P_{t-s} + = 0 +\]
+

where, in the second last equality, we used (27).

+

Hence \(V_s\) is constant, so our previous observations \(V_0 = \hat P_t\) and \(V_t = P_t\) +now yield \(\hat P_t = P_t\).

+

Since \(t\) was arbitrary, the proof is now done.

+
+
+
+ + + + +
+ +
+
+ + +
+ + The Markov Property + The Kolmogorov Forward Equation + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file diff --git a/kolmogorov_fwd.html b/kolmogorov_fwd.html new file mode 100644 index 0000000..b919cbb --- /dev/null +++ b/kolmogorov_fwd.html @@ -0,0 +1,905 @@ + + + + + + + + The Kolmogorov Forward Equation — Continuous Time Markov Chains + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + +
+ +
+ + + + + + + + +
+ +
+ + + + + + +
+
+ +
+ + +
+
+ Contents +
+ +
+
+
+
+ +
+ +
+

The Kolmogorov Forward Equation

+
+

Overview

+

In this lecture we approach continuous time Markov chains from a more +analytical perspective.

+

The emphasis will be on describing distribution flows through vector-valued +differential equations and their solutions.

+

These distribution flows show how the time \(t\) distribution associated with a +given Markov chain \((X_t)\) changes over time.

+

Distribution flows will be identified with initial value problems generated by autonomous linear ordinary differential equations (ODEs) in vector space.

+

We will see that the solutions of these flows are described by Markov semigroups.

+

This leads us back to the theory we have already constructed – some care will +be taken to clarify all the connections.

+

In order to avoid being distracted by technicalities, we continue to defer our +treatment of infinite state spaces, assuming throughout this lecture that \(|S| += n\).

+

As before, \(\dD\) is the set of all distributions on \(S\).

+

We will use the following imports

+
+
+
import numpy as np
+import scipy as sp
+import matplotlib.pyplot as plt
+import quantecon as qe
+from numba import njit
+from scipy.linalg import expm
+
+from matplotlib import cm
+from mpl_toolkits.mplot3d import Axes3D
+from mpl_toolkits.mplot3d.art3d import Poly3DCollection
+
+
+
+
+
+
+

From Difference Equations to ODEs

+

Previously we generated this figure, which shows how distributions evolve over time for the inventory model under a certain parameterization:

+
+
+_images/markov_prop_7_0.png +
+

Fig. 2 Probability flows for the inventory model.

+
+

(Hot colors indicate early dates and cool colors denote later dates.)

+

We also learned how this flow is related to +the Kolmogorov backward equation, which is an ODE.

+

In this section we examine distribution flows and their connection to +ODEs and continuous time Markov chains more systematically.

+
+

Review of the Discrete Time Case

+

Let \((X_t)\) be a discrete time Markov chain with Markov matrix \(P\).

+

Recall that, in the discrete time case, the distribution \(\psi_t\) of \(X_t\) updates according to

+
+\[ + \psi_{t+1} = \psi_t P, + \qquad \psi_0 \text{ a given element of } \dD, +\]
+

where distributions are understood as row vectors.

+

Here’s a visualization for the case \(|S|=3\), so that \(\dD\) is the unit +simplex in \(\RR^3\).

+

The initial condition is (0, 0, 1) and the Markov matrix is

+
+
+
P = ((0.9, 0.1, 0.0),
+     (0.4, 0.4, 0.2),
+     (0.1, 0.1, 0.8))
+
+
+
+
+
+
+
def unit_simplex(angle):
+    
+    fig = plt.figure(figsize=(8, 6))
+    ax = fig.add_subplot(111, projection='3d')
+
+    vtx = [[0, 0, 1],
+           [0, 1, 0], 
+           [1, 0, 0]]
+    
+    tri = Poly3DCollection([vtx], color='darkblue', alpha=0.3)
+    tri.set_facecolor([0.5, 0.5, 1])
+    ax.add_collection3d(tri)
+
+    ax.set(xlim=(0, 1), ylim=(0, 1), zlim=(0, 1), 
+           xticks=(1,), yticks=(1,), zticks=(1,))
+
+    ax.set_xticklabels(['$(1, 0, 0)$'], fontsize=12)
+    ax.set_yticklabels(['$(0, 1, 0)$'], fontsize=12)
+    ax.set_zticklabels(['$(0, 0, 1)$'], fontsize=12)
+
+    ax.xaxis.majorTicks[0].set_pad(15)
+    ax.yaxis.majorTicks[0].set_pad(15)
+    ax.zaxis.majorTicks[0].set_pad(35)
+
+    ax.view_init(30, angle)
+
+    # Move axis to origin
+    ax.xaxis._axinfo['juggled'] = (0, 0, 0)
+    ax.yaxis._axinfo['juggled'] = (1, 1, 1)
+    ax.zaxis._axinfo['juggled'] = (2, 2, 0)
+    
+    ax.grid(False)
+    
+    return ax
+
+
+def convergence_plot(ψ, n=14, angle=50):
+
+    ax = unit_simplex(angle)
+
+    P = ((0.9, 0.1, 0.0),
+         (0.4, 0.4, 0.2),
+         (0.1, 0.1, 0.8))
+    
+    P = np.array(P)
+    colors = cm.jet_r(np.linspace(0.0, 1, n))
+
+    x_vals, y_vals, z_vals = [], [], []
+    for t in range(n):
+        x_vals.append(ψ[0])
+        y_vals.append(ψ[1])
+        z_vals.append(ψ[2])
+        ψ = ψ @ P
+
+    ax.scatter(x_vals, y_vals, z_vals, c=colors, s=50, alpha=0.7, depthshade=False)
+
+    return ψ
+
+ψ = convergence_plot((0, 0, 1))
+
+plt.show()
+
+
+
+
+_images/kolmogorov_fwd_4_0.png +
+
+

There’s a sense in which a discrete time Markov chain “is” a homogeneous +linear difference equation in distribution space.

+

To clarify this, suppose we +take \(G\) to be a linear map from \(\dD\) to itself and +write down the difference equation

+
+(32)\[ + \psi_{t+1} = G(\psi_t) + \quad \text{with } \psi_0 \in \dD \text{ given}. +\]
+

Because \(G\) is a linear map from a finite dimensional space to itself, it can +be represented by a matrix.

+

Moreover, a matrix \(P\) is a Markov matrix if and only if \(\psi \mapsto +\psi P\) sends \(\dD\) into itself (check it if you haven’t already).

+

So, under the stated conditions, our difference equation (32) uniquely +identifies a Markov matrix, along with an initial condition \(\psi_0\).

+

Together, these objects identify the joint distribution of a discrete time Markov chain, as previously described.

+
+
+

Shifting to Continuous Time

+

We have just argued that a discrete time Markov chain can be identified with a +linear difference equation evolving in \(\dD\).

+

This strongly suggests that a continuous time Markov chain can be identified +with a linear ODE evolving in \(\dD\).

+

This intuition is correct and important.

+

The rest of the lecture maps out the main ideas.

+
+
+
+

ODEs in Distribution Space

+

Consider linear differential equation given by

+
+(33)\[ + \psi_t' = \psi_t Q, + \qquad \psi_0 \text{ a given element of } \dD, +\]
+

where

+
    +

  • \(Q\) is an \(n \times n\) matrix,

    • +

  • distributions are again understood as row vectors, and

    • +

  • derivatives are taken element by element, so that

    • +
+
+\[ + \psi_t' = + \begin{pmatrix} + \frac{d}{dt} \psi_t(x_1) & + \cdots & + \frac{d}{dt} \psi_t(x_n) + \end{pmatrix} +\]
+
+

Solutions to Linear Vector ODEs

+

Using the matrix exponential, the unique solution to the initial value problem +(33) is

+
+(34)\[ + \psi_t = \psi_0 P_t + \quad \text{where } P_t := e^{tQ} +\]
+

To check that (34) is a solution, we use (27) again to get

+
+\[ + \frac{d}{d t} P_t = Q e^{tQ} = e^{tQ} Q +\]
+

The first equality can be written as \(P_t' = Q P_t\) and this is just +the Kolmogorov backward equation.

+

The second equality can be written as

+
+\[ + P_t' = P_t Q +\]
+

and is called the Kolmogorov forward equation.

+

Applying the Kolmogorov forward equation, we obtain

+
+\[ + \frac{d}{d t} \psi_t + = \frac{d}{d t} \psi_0 P_t + = \psi_0 \frac{d}{d t} P_t + = \psi_0 P_t Q + = \psi_t Q +\]
+

This confirms that (34) solves (33).

+

Here’s an example of three distribution flows with dynamics generated by (33), one starting from each vertex.

+

The code uses (34) with matrix \(Q\) given by

+
+
+
Q = ((-3, 2, 1),
+     (3, -5, 2),
+     (4, 6, -10))
+
+
+
+
+
+
+
Q = np.array(Q)
+ψ_00 = np.array((0.01, 0.01, 0.99))
+ψ_01 = np.array((0.01, 0.99, 0.01))
+ψ_02 = np.array((0.99, 0.01, 0.01))
+
+ax = unit_simplex(angle=50)    
+
+def flow_plot(ψ, h=0.001, n=400, angle=50):
+    colors = cm.jet_r(np.linspace(0.0, 1, n))
+
+    x_vals, y_vals, z_vals = [], [], []
+    for t in range(n):
+        x_vals.append(ψ[0])
+        y_vals.append(ψ[1])
+        z_vals.append(ψ[2])
+        ψ = ψ @ expm(h * Q)
+
+    ax.scatter(x_vals, y_vals, z_vals, c=colors, s=20, alpha=0.2, depthshade=False)
+
+flow_plot(ψ_00)
+flow_plot(ψ_01)
+flow_plot(ψ_02)
+
+plt.show()
+
+
+
+
+_images/kolmogorov_fwd_7_0.png +
+
+

(Distributions cool over time, so initial conditions are hot colors.)

+
+
+

Forwards vs Backwards Equations

+

As the above discussion shows, we can take the Kolmogorov forward equation +\(P_t' = P_t Q\) and premultiply by any distribution \(\psi_0\) to get the +distribution ODE \(\psi'_t = \psi_t Q\).

+

In this sense, we can understand the Kolmogorov forward equation as pushing +distributions forward in time.

+

Analogously, we can take the Kolmogorov backward equation +\(P_t' = Q P_t\) and postmultiply by any vector \(h\) to get

+
+\[ + (P_t h)' = Q P_t h +\]
+

Recalling that \((P_t h)(x) = \EE [ h(X_t) \,|\, X_0 = x]\), this vector +ODE tells us how expectations evolve, conditioning backward to time zero.

+

Both the forward and the backward equations uniquely pin down the same solution \(P_t = e^{tQ}\) when combined with the initial condition \(P_0 = I\).

+
+
+

Matrix- vs Vector-Valued ODEs

+

The ODE \(\psi'_t = \psi_t Q\) is sometimes called the +Fokker–Planck equation (although this terminology is most commonly used +in the context of diffusions).

+

It is a vector-valued ODE that describes the evolution of a particular +distribution path.

+

By comparison, the Kolmogorov forward equation is (like the backward equation) +a differential equation in matrices.

+

(And matrices are really maps, that send vectors into vectors.)

+

Operating at this level is less intuitive and more abstract than working with the +Fokker–Planck equation.

+

But, in the end, the object that we want to describe is a the Markov +semigroup.

+

The Kolmogorov forward and backward equations are the ODEs that define +this fundamental object.

+
+
+

Preserving Distributions

+

In the simulation above, \(Q\) was chosen with some care, so that the flow +remains in \(\dD\).

+

What are the exact properties we require on \(Q\) such that \(\psi_t\) is always +in \(\dD\)?

+

This is an important question, because we are setting up an exact +correspondence between linear ODEs that evolve in \(\dD\) and continuous +time Markov chains.

+

Recall that the linear update rule \(\psi \mapsto \psi P\) is invariant on +\(\dD\) if and only if \(P\) is a Markov matrix.

+

So now we can rephrase our key question regarding invariance on \(\dD\):

+

What properties do we need to impose on \(Q\) so that \(P_t = e^{tQ}\) is a Markov matrix +for all \(t\)?

+

A square matrix \(Q\) is called an intensity matrix if \(Q\) has zero row +sums and \(Q(x, y) \geq 0\) whenever \(x \not= y\).

+
+

Theorem 10

+
+

If \(Q\) is a matrix on \(S\) and \(P_t := e^{tQ}\), then the following statements +are equivalent:

+
    +

  1. \(P_t\) is a Markov matrix for all \(t\).

    2. +

  2. \(Q\) is an intensity matrix.

    3. +
+
+

The proof is related to that of Lemma 9 and is found as +a solved exercise below.

+
+

Corollary 11

+
+

If \(Q\) is an intensity matrix on finite \(S\) and \(P_t = e^{tQ}\) for all \(t \geq 0\), +then \((P_t)\) is a Markov semigroup.

+
+

We call \((P_t)\) the Markov semigroup generated by \(Q\).

+

Later we will see that this result extends to the case \(|S| = \infty\) under +some mild restrictions on \(Q\).

+
+
+
+

Jump Chains

+

Let’s return to the chain \((X_t)\) created from jump chain pair \((\lambda, K)\) in +Algorithm 6.

+

We found that the semigroup is given by

+
+\[ + P_t = e^{tQ} + \quad \text{where} \quad + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +\]
+

Using the fact that \(K\) is a Markov matrix and the jump rate function +\(\lambda\) is nonnegative, you can easily check that \(Q\) satisfies the +definition of an intensity matrix.

+

Hence \((P_t)\), the Markov semigroup for the jump chain \((X_t)\), is the +semigroup generated by the intensity matrix \(Q(x, y) = \lambda(x) (K(x, y) - I(x, y))\).

+

We can differentiate \(P_t = e^{tQ}\) to obtain the Kolmogorov forward equation +\(P_t' = P_t Q\).

+

We can then premultiply by \(\psi_0 \in \dD\) to get \(\psi_t' = \psi_t +Q\), which is the Fokker–Planck equation.

+

More explicitly, for given \(y \in S\),

+
+\[ + \psi_t'(y) + = \sum_{x \not= y} \psi_t(x) \lambda(x) K(x, y) - \psi_t(y) \lambda(y) +\]
+

The rate of probability flow into \(y\) is equal to the inflow from other states +minus the outflow.

+
+
+

Summary

+

We have seen that any intensity matrix \(Q\) on \(S\) defines a Markov semigroup via \(P_t = e^{tQ}\).

+

Henceforth, we will say that \((X_t)\) is a Markov chain with intensity matrix \(Q\) if +\((X_t)\) is a Markov chain with Markov semigroup \((e^{tQ})\).

+

While our discussion has been in the context of a finite state space, later we +will see that these ideas carry over to an infinite state setting under mild +restrictions.

+

We have also hinted at the fact that every continuous time Markov chain +is a Markov chain with intensity matrix \(Q\) for some suitably chosen \(Q\).

+

Later we will prove this to be universally true when \(S\) is finite and true +under mild conditions when \(S\) is countably infinite.

+

Intensity matrices are important because

+
    +

  1. they are the natural infinitesimal descriptions of Markov semigroups,

    2. +

  2. they are often easy to write down in applications and

    3. +

  3. they provide an intuitive description of dynamics.

    4. +
+

Later, we will see that, for a given intensity matrix \(Q\), the elements are +understood as follows:

+
    +

  • when \(x \not= y\), the value \(Q(x, y)\) is the “rate of leaving \(x\) for \(y\)” and

    • +

  • \(-Q(x, x) \geq 0\) is the “rate of leaving \(x\)” .

    • +
+
+
+

Exercises

+
+

Exercise 1

+

Let \((P_t)\) be a Markov semigroup such that \(t \mapsto P_t(x, y)\) is +differentiable at all \(t \geq 0\) and \((x, y) \in S \times S\).

+

(The derivative at \(t=0\) is the usual right derivative.)

+

Define (pointwise, at each \((x,y)\)),

+
+(35)\[ + Q := P'_0 = \lim_{h \downarrow 0} \frac{P_h - I}{h} +\]
+

Assuming that this limit exists, and hence \(Q\) is well-defined, show that

+
+\[ + P'_t = P_t Q + \quad \text{and} \quad + P'_t = Q P_t +\]
+

both hold. (These are the Kolmogorov forward and backward equations.)

+
+
+

Exercise 2

+

Recall our model of jump chains with state-dependent jump +intensities given by rate function \(x \mapsto \lambda(x)\).

+

After a wait time with exponential rate \(\lambda(x) \in (0, \infty)\), the +state transitions from \(x\) to \(y\) with probability \(K(x,y)\).

+

We found that the associated semigroup \((P_t)\) satisfies the Kolmogorov +backward equation \(P'_t = Q P_t\) with

+
+(36)\[ + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +\]
+

Show that \(Q\) is an intensity matrix and that (35) holds.

+
+
+

Exercise 3

+

Prove Theorem 10 by adapting the arguments in Lemma 9. +(This is nontrivial but worth at least trying.)

+

Hint: The constant \(m\) in the proof can be set to \(\max_x |Q(x, x)|\).

+
+
+
+

Solutions

+
+

Solution to Exercise 1

+

Let \((P_t)\) be a Markov semigroup and let \(Q\) be as defined in the statement +of the exercise.

+

Fix \(t \geq 0\) and \(h > 0\).

+

Combining the semigroup property and linearity with the restriction \(P_0 = I\), we get

+
+\[ + \frac{P_{t+h} - P_t}{h} + = \frac{P_t P_h - P_t}{h} + = \frac{P_t (P_h - I)}{h} +\]
+

Taking \(h \downarrow 0\) and using the definition of \(Q\) gives \(P_t' = P_t Q\), +which is the Kolmogorov forward equation.

+

For the backward equation we observe that

+
+\[ + \frac{P_{t+h} - P_t}{h} + = \frac{P_h P_t - P_t}{h} + = \frac{(P_h - I) P_t}{h} +\]
+

also holds. Taking \(h \downarrow 0\) gives the Kolmogorov backward equation.

+
+
+

Solution to Exercise 2

+

Let \(Q\) be as defined in (36).

+

We need to show that \(Q\) is nonnegative off the diagonal and has zero row +sums.

+

The first assertion is immediate from nonnegativity of \(K\) and \(\lambda\).

+

For the second, we use the fact that \(K\) is a Markov matrix, so that, with \(1\) +as a column vector of ones,

+
+\[ + Q 1 + = \lambda (K 1 - 1) + = \lambda (1 - 1) + = 0 +\]
+
+
+

Solution to Exercise 3

+

Suppose that \(Q\) is an intensity matrix, fix \(t \geq 0\) and set \(P_t = e^{tQ}\).

+

The proof from Lemma 9 that \(P_t\) has unit row sums applies +directly to the current case.

+

The proof of nonnegativity of \(P_t\) can be applied after some +modifications.

+

To this end, set \(m := \max_x |Q(x,x)|\) and \(\hat P := I + Q / m\).

+

You can check that \(\hat P\) is a Markov matrix and that \(Q = m( \hat P - I)\).

+

The rest of the proof of nonnegativity of \(P_t\) is unchanged and we will not repeat it.

+

We conclude that \(P_t\) is a Markov matrix.

+

Regarding the converse implication, suppose that \(P_t = e^{tQ}\) is a Markov +matrix for all \(t\) and let \(1\) be a column vector of ones.

+

Because \(P_t\) has unit row sums and differentiation is linear, +we can employ the Kolmogorov backward equation to obtain

+
+\[ + Q 1 + = Q P_t 1 + = \left( \frac{d}{d t} P_t \right) 1 + = \frac{d}{d t} (P_t 1) + = \frac{d}{d t} 1 + = 0 +\]
+

Hence \(Q\) has zero row sums.

+

We can use the definition of the matrix exponential to obtain, +for any \(x, y\) and \(t \geq 0\),

+
+(37)\[ + P_t(x, y) = \mathbb 1\{x = y\} + t Q(x, y) + o(t) +\]
+

From this equality and the assumption that \(P_t\) is a Markov matrix for all +\(t\), we see that the off diagonal elements of \(Q\) must be +nonnegative.

+

Hence \(Q\) is a intensity matrix.

+
+
+
+ + + + +
+ +
+
+ + +
+ + Kolmogorov Backward Equation + Semigroups and Generators + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file diff --git a/markov_prop.html b/markov_prop.html new file mode 100644 index 0000000..e05494c --- /dev/null +++ b/markov_prop.html @@ -0,0 +1,1349 @@ + + + + + + + + The Markov Property — Continuous Time Markov Chains + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + +
+ +
+ + + + + + + + +
+ +
+ + + + + + +
+
+ +
+ + +
+
+ Contents +
+ +
+
+
+
+ +
+ +
+

The Markov Property

+
+

Overview

+

A continuous time stochastic process is said to have the Markov property if +that the past and future are independent given the current state.

+

(A more formal definition is provided below.)

+

As we will see, the Markov property imposes a large amount of structure on +continuous time processes.

+

This structure leads to elegant and powerful results on +evolution and dynamics.

+

At the same time, the Markov property is general enough to cover many applied +problems, as described in the introduction.

+
+

Setting

+

In this lecture, the state space where dynamics +evolve will be a countable set, +denoted henceforth by \(S\), with typical elements \(x, y\).

+

(Note that “countable” is understood to include finite.)

+

Regarding notation, in what follows, \(\sum_{x \in S}\) is abbreviated to +\(\sum_x\), the supremum \(\sup_{x \in S}\) is abbreviated to \(\sup_x\) and so on.

+

A distribution on \(S\) is a function \(\phi\) from \(S\) to \(\RR_+\) with +\(\sum_x \phi(x) = 1\).

+

Let \(\dD\) denote the set of all distributions on \(S\).

+

To economize on terminology, we define a matrix \(A\) on \(S\) to be a map +from \(S \times S\) to \(\RR\).

+

When \(S\) is finite, this reduces to the usual notion of a matrix, and, +whenever you see expressions such as \(A(x,y)\) below, you can +mentally identify them with more familiar matrix +notation, such as \(A_{ij}\), if you wish.

+

The product of two matrices \(A\) and \(B\) is defined by

+
+(9)\[ + (A B)(x, y) = \sum_z A(x, z) B(z, y) + \qquad ((x, y) \in S \times S) +\]
+

If \(S\) is finite, then this is just ordinary matrix multiplication.

+

In statements involving matrix algebra, we always treat distributions as row +vectors, so that, for \(\phi \in \dD\) and given matrix \(A\),

+
+\[ + (\phi A)(y) = \sum_x \phi(x) A(x, y) +\]
+

We will use the following imports

+
+
+
import numpy as np
+import scipy as sp
+import matplotlib.pyplot as plt
+
+import quantecon as qe
+from numba import njit
+
+from scipy.linalg import expm
+from scipy.stats import binom
+
+from matplotlib import cm
+from mpl_toolkits.mplot3d import Axes3D
+
+
+
+
+
+
+
+

Markov Processes

+

We now introduce the definition of Markov processes, first reviewing the +discrete case and then shifting to continuous time.

+
+

Discrete Time, Finite State

+

The simplest Markov processes are those with a discrete time parameter and finite state space.

+

Assume for now that \(S\) has \(n\) elements and let \(P\) be a Markov matrix, +which means that \(P(x,y) \geq 0\) and \(\sum_y P(x,y) = 1\) for all \(x\).

+

In applications, \(P(x, y)\) represents the probability of transitioning from \(x\) to +\(y\) in one step.

+

A Markov chain \((X_t)_{t \in \ZZ_+}\) on \(S\) with Markov +matrix \(P\) is a sequence of random variables satisfying

+
+(10)\[ + \PP\{X_{t+1} = y \,|\, X_0, X_1, \ldots, X_t \} = P (X_t, y) +\]
+

with probability one for all \(y \in S\) and any \(t \in \ZZ_+\).

+

In addition to connecting probabilities to the Markov matrix, +(10) says that the process depends on its history only through +the current state.

+

We recall that, if \(X_t\) +has distribution \(\phi\), then \(X_{t+1}\) has distribution \(\phi P\).

+

Since \(\phi\) is understood as a row vector, the meaning is

+
+(11)\[ + (\phi P)(y) = \sum_x \phi(x) P(x, y) + \qquad (y \in S) +\]
+
+

The Joint Distribution

+

In general, for given Markov matrix \(P\), there can be many Markov chains +\((X_t)\) that satisfy (10).

+

This is due to the more general observation that, for a given distribution +\(\phi\), we can construct many random variables having distribution \(\phi\).

+

(The exercises below ask for one example.)

+

Hence \(P\) is, in a sense, a more primitive object than \((X_t)\).

+

There is another way to see the fundamental importance of \(P\), which is by +constructing the joint distribution of \((X_t)\) from \(P\).

+

Let \(S^\infty\) represent the space of \(S\)-valued sequences \((x_0, x_1, x_2, \ldots)\).

+

Fix an initial condition \(\psi \in \dD\) and a Markov matrix \(P\) on \(S\).

+

The joint distribution of a Markov chain \((X_t)\) satisfying +(10) and \(X_0 \sim \psi\) is the distribution \(\mathbf P_\psi\) over +\(S^\infty\) such that

+
+(12)\[ + \PP\{ X_{t_1} = y_1, \ldots, X_{t_k} = y_k \} + = + \mathbf P_\psi\{ (x_t) \in S^\infty \,:\, + x_{t_i} = y_i \text{ for } i = 1, \ldots m\} +\]
+

for any \(m\) positive integers \(t_i\) and \(m\) elements \(y_i\) of the state space \(S\).

+

(Joint distributions of discrete time processes are uniquely defined by their +values at finite collections of times — see, for example, Theorem 7.2 of [Wal12].)

+

We can construct \(\mathbf P_\psi\) by first defining \(P_\psi^n\) over +the the finite Cartiesian product \(S^{n+1}\) via

+
+(13)\[ + \mathbf P_\psi^n(x_0, x_1, \ldots, x_n) + = \psi(x_0) + P(x_0, x_1) + \times \cdots \times + P(x_{n-1}, x_n) +\]
+

For any Markov chain \((X_t)\) satisfying (10) and \(X_0 \sim \psi\), +the restriction \((X_0, \ldots, X_n)\) has joint distribution \(\mathbf +P_\psi^n\).

+

This is a solved exercise below.

+

The last step is to show that the family \((\mathbf P_\psi^n)\) defined at each +\(n \in \NN\) extends uniquely to a distribution \(\mathbf P_\psi\) over the +infinite sequences in \(S^\infty\).

+

That this is true follows from a well known theorem of Kolmogorov.

+

Hence \(P\) defines the joint distribution \(\mathbf P_\psi\) when paired with any initial condition \(\psi\).

+
+
+
+

Extending to Countable State Spaces

+

When \(S\) is infinite, the same idea carries through.

+

Consistent with the finite case, a Markov matrix is a map +\(P\) from \(S \times S\) to \(\RR_+\) satisfying

+
+\[ + \sum_y P(x, y) = 1 + \text{ for all } x \in S +\]
+

The definition of a Markov chain \((X_t)_{t \in \ZZ_+}\) on \(S\) with Markov matrix \(P\) is exactly as in (10).

+

Given Markov matrix \(P\) and \(\phi \in \dD\), we define \(\phi P\) by +(11).

+

Then, as before, \(\phi P\) can be understood as the distribution of +\(X_{t+1}\) when \(X_t\) has distribution \(\phi\).

+

The function \(\phi P\) is in \(\dD\), since, by (11), it is +nonnegative and

+
+\[ + \sum_y (\phi P)(y) + = \sum_y \sum_x P(x, y) \phi(x) + = \sum_x \sum_y P(x, y) \phi(x) + = \sum_x \phi(x) + = 1 +\]
+

(Swapping the order of infinite sums is justified here by the fact that all +elements are nonnegative — a version of Tonelli’s theorem).

+

If \(P\) and \(Q\) are Markov matrices on \(S\), then, using the definition in +(9),

+
+\[ + (P Q)(x, y) := \sum_z P(x, z) Q(z, y) +\]
+

It is not difficult to check that \(P Q\) is again a Markov matrix on \(S\).

+

The elements of \(P^k\), the \(k\)-th product of \(P\) with itself, give \(k\) step transition probabilities.

+

For example, we have

+
+(14)\[ + P^k(x, y) + = (P^{k-j} P^j)(x, y) = \sum_z P^{k-j}(x, z) P^j(z, y) +\]
+

which is a version of the (discrete time) Chapman-Kolmogorov equation.

+

Equation (14) can be obtained from the law of total probability: if +\((X_t)\) is a Markov chain with Markov matrix \(P\) and initial condition \(X_0 = +x\), then

+
+\[ + \PP\{X_k = y\} + = \sum_z \PP\{X_k = y \,|\, X_j=z\} \PP\{X_j=z\} +\]
+

All of the preceding discussion on the connection between \(P\) +and the joint distribution of \((X_t)\) when \(S\) is finite carries over +to the current setting.

+
+
+

The Continuous Time Case

+

A continuous time stochastic process on \(S\) is a collection \((X_t)\) of \(S\)-valued +random variables \(X_t\) defined on a common probability space and indexed by \(t +\in \RR_+\).

+

Let \(I\) be the Markov matrix on \(S\) defined by \(I(x,y) = \mathbb 1\{x = y\}\).

+

A Markov semigroup is a family \((P_t)\) of Markov matrices +on \(S\) satisfying

+
    +

  1. \(P_0 = I\),

    2. +

  2. \(\lim_{t \to 0} P_t(x, y) = I(x,y)\) for all \(x,y\) in \(S\), and

    3. +

  3. the semigroup property \(P_{s + t} = P_s P_t\) for all \(s, t \geq 0\).

    4. +
+

The interpretation of \(P_t(x, y)\) is the probability of moving from state \(x\) +to state \(y\) in \(t\) units of time.

+

As such it is natural that \(P_0(x,y) = 1\) if \(x=y\) and zero otherwise, which +is condition 1.

+

Condition 2 is continuity with respect to \(t\), which might seem restrictive +but it is in fact very mild.

+

For all practical applications, probabilities do not jump — although the +chain \((X_t)\) itself can of course jump from state to state as time +goes by.1

+

The semigroup property in condition 3 is nothing more than a continuous +time version of the Chapman-Kolmogorov equation.

+

This becomes clearer if we write it more explicitly as

+
+(15)\[ + P_{s+t}(x, y) + = \sum_z P_s(x, z) P_t(z, y) +\]
+

A stochastic process \((X_t)\) is called a (time homogeneous) continuous time +Markov chain on \(S\) with Markov semigroup \((P_t)\) if

+
+(16)\[ + \PP\{X_{s + t} = y \,|\, \fF_s \} + = P_t (X_s, y) +\]
+

with probability one for all \(y \in S\) and \(s, t \geq 0\).

+

Here \(\fF_s\) is the history \((X_r)_{r \leq s}\) of the process up until +time \(s\).

+

If you are an economist you might call \(\fF_s\) the “information set” at time +\(s\).

+

If you are familiar with measure theory, you can understand \(\fF_s\) as +the \(\sigma\)-algebra generated by \((X_r)_{r \leq s}\).

+

Analogous to the discrete time case, the joint +distribution of \((X_t)\) is determined by its Markov semigroup plus an +initial condition.

+

This distribution is defined over the set of all right continuous functions +\(\RR_+ \ni t \mapsto x_t \in S\), which we call \(rcS\).

+

Next one builds finite dimensional distributions over \(rcS\) using +expressions similar to (13).

+

Finally, the Kolmogorov extension theorem is applied, similar to the discrete +time case.

+

Corollary 6.4 of [LG16] provides full details.

+
+
+

The Canonical Chain

+

Given a Markov semigroup \((P_t)\) on \(S\), does there always exist a continuous +time Markov chain \((X_t)\) such that (16) holds?

+

The answer is affirmative.

+

To illustrate, pick any Markov semigroup \((P_t)\) on \(S\) and fix initial +condition \(\psi\).

+

Next, create the corresponding joint distribution \(\mathbf P_\psi\) over +\(rcS\), as described above.

+

Now, for each \(t \geq 0\), let \(\pi_t\) be the point evaluation functional on +\(rcS\), that maps right continuous function \((x_\tau)\) into its time \(t\) value +\(x_t\).

+

Finally, let \(X_t\) be an \(S\)-valued function on \(rcS\) defined at \((x_\tau) \in rcS\) by \(\pi_t ( (x_\tau))\).

+

In other words, after \(\mathbf P_\psi\) picks out some time path \((x_\tau) \in +rcS\), the Markov chain \((X_t)\) simply reports this time path.

+

Hence \((X_t)\) automatically has the correct distribution.

+

The chain \((X_t)\) constructed in this way is called the canonical chain +for the semigroup \((P_t)\) and initial condition \(\psi\).

+
+
+

Simulation and Probabilistic Constructions

+

While we have answered the existence question in the affirmative, +the canonical construction is quite abstract.

+

Moreover, there is little information about how we might simulate such a chain.

+

Fortunately, it turns out that there are more concrete ways to build +continuous time Markov chains from the objects that describe their +distributions.

+

We will learn about these in a later lecture.

+
+
+
+

Implications of the Markov Property

+

The Markov property carries some strong implications that are not immediately +obvious.

+

Let’s take some time to explore them.

+
+

Example: Failure of the Markov Property

+

Let’s look at how the Markov property can fail, via an intuitive rather than +formal discussion.

+

Let \((X_t)\) be a continuous time stochastic process with state space \(S = \{0, 1\}\).

+

The process starts at \(0\) and updates at follows:

+
    +

  1. Draw \(W\) independently from a fixed Pareto distribution.

    2. +

  2. Hold \((X_t)\) in its current state for \(W\) units of time and then switch +to the other state.

    3. +

  3. Go to step 1.

    4. +
+

What is the probability that \(X_{s+h} = i\) given both the history \((X_r)_{r \leq s}\) and current information \(X_s = i\)?

+

If \(h\) is small, then this is close to the +probability that there are zero switches over the time interval \((s, s+h]\).

+

To calculate this probability, it would be helpful to know how long the +state has been at current state \(i\).

+

This is because the Pareto distribution is not memoryless.

+

(With a Pareto distribution, if we know that \(X_t\) has been at \(i\) for a long +time, then a switch in the near future becomes more likely.)

+

As a result, the history prior to \(X_s\) is useful for predicting \(X_{s+h}\), +even when we know \(X_s\).

+

Thus, the Markov property fails.

+
+
+

Restrictions Imposed by the Markov Property

+

From the discussion above, we see that, for continuous time Markov chains, +the length of time between jumps must be memoryless.

+

Recall that, by Theorem 1, the only memoryless +distribution supported on \(\RR_+\) is the exponential distribution.

+

Hence, a continuous time Markov chain waits at states for an +exponential amount of time and then jumps.

+

The way that the new state is chosen must also satisfy the Markov property, +which adds another restriction.

+

In summary, we already understand the following about continuous time Markov chains:

+
    +

  1. Holding times are independent exponential draws.

    2. +

  2. New states are chosen in a ``Markovian’’ way, independent of the past given the current state.

    3. +
+

We just need to clarify the details in these steps to have a complete description.

+
+
+
+

Examples of Markov Processes

+

Let’s look at some examples of processes that possess the Markov property.

+
+

Example: Poisson Processes

+

The Poisson process discussed in our previous lecture is a +Markov process on state space \(\ZZ_+\).

+

To obtain the Markov semigroup, we observe that, for \(k \geq j\),

+
+\[ + \PP\{N_{s + t} = k \,|\, N_s = j\} + = \PP\{N_{s + t} - N_s = k - j \,|\, N_s = j\} + = \PP\{N_{s + t} - N_s = k - j\} +\]
+

where the last step is due to independence of increments.

+

From stationarity of increments we have

+
+\[ + \PP\{N_{s + t} - N_s = k - j\} + = \PP\{N_t = k - j\} + = e^{-\lambda t} \frac{ (\lambda t)^{k-j} }{(k-j)!} +\]
+

In summary, the Markov semigroup is

+
+(17)\[ + P_t(j, k) + = e^{-\lambda t} \frac{ (\lambda t)^{k-j} }{(k-j)!} +\]
+

whenever \(j \leq k\) and \(P_t(j, k) = 0\) otherwise.

+

This chain of equalities was obtained with \(N_s = j\) for arbitrary \(j\), so we +can replace \(j\) with \(N_s\) in (17) to verify the Markov property (16) for the Poisson process.

+

Under (17), each \(P_t\) is a Markov matrix and \((P_t)\) is a +Markov semigroup.

+

The proof of the semigroup property is a solved exercise below.2

+
+
+
+

A Model of Inventory Dynamics

+

Let \(X_t\) be the inventory of a firm at time \(t\), taking values in the +integers \(0, 1, \ldots, b\).

+

If \(X_t > 0\), then a customer arrives after \(W\) +units of time, where \(W \sim E(\lambda)\) for some fixed \(\lambda > 0\).

+

Upon arrival, each customer purchases \(\min\{U, X_t\}\) units, where \(U\) is an +IID draw from the geometric distribution started at 1 rather than 0:

+
+\[ + \PP\{U = k\} = (1-\alpha)^{k-1} \alpha + \qquad (k = 1, 2, \ldots, \; \alpha \in (0, 1)) +\]
+

If \(X_t = 0\), then no customers arrive and the firm places an order for \(b\) units.

+

The order arrives after a delay of \(D\) units of time, where \(D \sim E(\lambda)\).

+

(We use the same \(\lambda\) here just for convenience, to simplify the exposition.)

+
+

Representation

+

The inventory process jumps to a new value either when a new customer arrives +or when new stock arrives.

+

Between these arrival times it is constant.

+

Hence, to track \(X_t\), it is enough to track the jump times and the new values +taken at the jumps.

+

In what follows, we denote the jump times by \(\{J_k\}\) and the values at jumps +by \(\{Y_k\}\).

+

Then we construct the state process via

+
+(18)\[ + X_t = \sum_{k \geq 0} Y_k \mathbb 1\{J_k \leq t < J_{k+1}\} + \qquad (t \geq 0) +\]
+
+
+

Simulation

+

Let’s simulate this process, starting at \(X_0 = 0\).

+

As above,

+
    +

  • \(J_k\) is the time of the \(k\)-th jump (up or down) in inventory.

    • +

  • \(Y_k\) is the size of the inventory after the \(k\)-th jump.

    • +

  • \((X_t)\) is defined from these objects via (18).

    • +
+

Here’s a function that generates and returns one path \(t \mapsto X_t\).

+

(We are not aiming for computational efficiency at this stage.)

+
+
+
def sim_path(T=10, seed=123, λ=0.5, α=0.7, b=10):
+    """
+    Generate a path for inventory starting at b, up to time T.
+
+    Return the path as a function X(t) constructed from (J_k) and (Y_k).
+    """
+
+    J, Y = 0, b
+    J_vals, Y_vals = [J], [Y]
+    np.random.seed(seed)
+
+    while True:
+        W = np.random.exponential(scale=1/λ)  # W ~ E(λ)
+        J += W
+        J_vals.append(J)
+        if J >= T:
+            break
+        # Update Y
+        if Y == 0:
+            Y = b
+        else:
+            U = np.random.geometric(α)
+            Y = Y - min(Y, U)
+        Y_vals.append(Y)
+    
+    Y_vals = np.array(Y_vals)
+    J_vals = np.array(J_vals)
+
+    def X(t):
+        if t == 0.0:
+            return Y_vals[0]
+        else:
+            k = np.searchsorted(J_vals, t)
+            return Y_vals[k-1]
+
+    return X
+
+
+
+
+

Let’s plot the process \((X_t)\).

+
+
+
T = 20
+X = sim_path(T=T)
+
+grid = np.linspace(0, T, 100)
+
+fig, ax = plt.subplots()
+ax.step(grid, [X(t) for t in grid], label="$X_t$")
+
+ax.set(xlabel="time", ylabel="inventory")
+
+ax.legend()
+plt.show()
+
+
+
+
+_images/markov_prop_5_0.png +
+
+

As expected, inventory falls and then jumps back up to \(b\).

+
+
+

The Embedded Jump Chain

+

In models such as the one described above, the embedded discrete time +process \((Y_k)\) is called the “embedded jump chain”.

+

It is easy to see that \((Y_k)\) is discrete time finite state Markov chain.

+

Its Markov matrix \(K\) is +given by \(K(x, y) = \mathbb 1\{y=b\}\) when \(x=0\) and, when \(0 < x \leq b\),

+
+(19)\[\begin{split} + K(x, y) + = + \begin{cases} + \mathbb 0 & \text{ if } y \geq x + \\ + \PP\{x - U = y\} = (1-\alpha)^{x-y-1} \alpha + & \text{ if } 0 < y < x + \\ + \PP\{U \geq x\} = (1-\alpha)^{x-1} + & \text{ if } y = 0 + \end{cases} +\end{split}\]
+
+
+

Markov Property

+

The inventory model just described has the Markov property precisely because

+
    +

  1. the jump chain \((Y_k)\) is Markov in discrete time and

    2. +

  2. the holding times are independent exponential draws.

    3. +
+

Rather than providing more details on these points here, let us first describe +a more general setting where the arguments will be clearer and more useful.

+
+
+
+

Jump Processes with Constant Rates

+

The examples we have focused on so far are special cases of Markov processes +with constant jump intensities.

+

This processes turn out to be very representative (although the constant jump intensity will later be relaxed).

+

Let’s now summarize the model and its properties.

+
+

Construction

+

The data for a Markov process on \(S\) with constant jump rates are

+
    +

  • a parameter \(\lambda > 0\) called the jump rate, which governs the jump +intensities and

    • +

  • a Markov matrix \(K\) on \(S\), called the jump matrix.

    • +
+

To run the process we also need an initial condition \(\psi \in \dD\).

+

The process \((X_t)\) is constructed by holding at each state for an +exponential amount of time, with rate \(\lambda\), and then updating to a +new state via \(K\).

+

In more detail, the construction is

+
+

Algorithm 5 (Constant Rate Jump Chain)

+
+

Inputs \(\psi \in \dD\), positive constant \(\lambda\), Markov matrix \(K\)

+

Outputs Markov chain \((X_t)\)

+
    +

  1. draw \(Y_0\) from \(\psi\)

    2. +

  2. set \(k = 1\) and \(J_0 = 0\)

    3. +

  3. draw \(W_k\) from Exp\((\lambda)\) and set \(J_k = J_{k-1} + W_k\)

    4. +

  4. set \(X_t = Y_{k-1}\) for all \(t\) such that \(J_{k-1} \leq t < J_k\).

    5. +

  5. draw \(Y_k\) from \(K(Y_{k-1}, \cdot)\)

    6. +

  6. set \(k = k+1\) and go to step 3.

    7. +
+
+

An alternative, more parsimonious way to express the same process is to take

+
    +

  • \((N_t)\) to be a Poisson process with rate \(\lambda\) and

    • +

  • \((Y_k)\) to be a discrete time Markov chain with Markov matrix \(K\)

    • +
+

and then set

+
+\[ + X_t := Y_{N_t} \text{ for all } t \geq 0 +\]
+

As before, the discrete time process \((Y_k)\) is called the embedded jump chain.

+

(Not to be confused with \((X_t)\), which is often called a “jump process” or +“jump chain” due to the fact that it changes states with jumps.)

+

The draws \((W_k)\) are called the wait times or holding times.

+
+
+

Examples

+

The Poisson process with rate \(\lambda\) is a jump process on \(S = \ZZ_+\).

+

The holding times are obviously exponential with constant rate \(\lambda\).

+

The jump matrix is just \(K(i, j) = \mathbb 1\{j = i+1\}\), so that the state +jumps up by one at every \(J_k\).

+

The inventory model is also a jump process with constant rate \(\lambda\), this +time on \(S = \{0, 1, \ldots, b\}\).

+

The jump matrix was given in (19).

+
+
+

Markov Property

+

Let’s show that the jump process \((X_t)\) constructed above satisfies the +Markov property, and obtain the Markov semigroup at the same time.

+

We will use two facts:

+
    +

  • the jump chain \((Y_k)\) has the Markov property in discrete +time and

    • +

  • the Poisson process has stationary independent increments.

    • +
+

From these facts it is intuitive that the distribution of \(X_{t+s}\) given +the whole history \(\fF_s = \{ (N_r)_{r \leq s}, (Y_k)_{k \leq N_s} \}\) +depends only on \(X_s\).

+

Indeed, if we know \(X_s\), then we can simply

+
    +

  • restart the Poisson process from \(N_s\) and then

    • +

  • starting from \(X_s = Y_{N_s}\), update the embedded jump chain \((Y_k)\) using \(K\) each time a new jump occurs.

    • +
+

Let’s write this more mathematically.

+

Fixing \(y \in S\) and \(s, t \geq 0\), we have

+
+\[ + \PP\{X_{s + t} = y \,|\, \fF_s \} + = \PP\{Y_{N_{s + t}} = y \,|\, \fF_s \} + = \PP\{Y_{N_s + N_{s + t} - N_s} = y \,|\, \fF_s \} +\]
+

Recalling that \(N_{s + t} - N_s\) is Poisson distributed with rate \(t \lambda\), independent of the history \(\fF_s\), we can write the display above as

+
+\[ + \PP\{X_{s + t} = y \,|\, \fF_s \} + = + \sum_{k \geq 0} + \PP\{Y_{N_s + k} = y \,|\, \fF_s \} + \frac{(t \lambda )^k}{k!} e^{-t \lambda} +\]
+

Because the embedded jump chain is Markov with Markov matrix \(K\), we can simplify further to

+
+\[ + \PP\{X_{s + t} = y \,|\, \fF_s \} + = \sum_{k \geq 0} + K^k(Y_{N_s}, y) \frac{(t \lambda )^k}{k!} e^{-t \lambda} + = K^k(X_s, y) \frac{(t \lambda )^k}{k!} e^{-t \lambda} +\]
+

Since the expression above depends only on \(X_s\), +we have proved that \((X_t)\) has the Markov property.

+
+
+

Transition Semigroup

+

The Markov semigroup can be obtained from our final result, conditioning +on \(X_s = x\) to get

+
+\[ + P^t(x, y) = \PP\{X_{s + t} = y \,|\, X_s = x \} + = e^{-t \lambda} \sum_{k \geq 0} + K^k(x, y) \frac{(t \lambda )^k}{k!} +\]
+

If \(S\) is finite, we can write this in matrix form and use the definition of +the matrix exponential to +get

+
+\[ + P^t + = e^{-t \lambda} + \sum_{k \geq 0} + \frac{(t \lambda K)^k}{k!} + = e^{-t \lambda} e^{t \lambda K} + = e^{t \lambda (K - I)} +\]
+

This is a simple and elegant representation of the Markov semigroup that +makes it easy to understand and analyze distribution dynamics.

+

For example, if \(X_0\) has distribution \(\psi\), then \(X_t\) has distribution

+
+(20)\[ + \psi P_t = \psi e^{t \lambda (K - I)} +\]
+

We just need to plug in \(\lambda\) and \(K\) to obtain the entire flow \(t \mapsto \psi P_t\).

+

We will soon extend this representation to the case where \(S\) is infinite.

+
+
+
+

Distribution Flows for the Inventory Model

+

Let’s apply these ideas to the inventory model described above.

+

We fix

+
    +

  • the parameters \(\alpha\), \(b\) and \(\lambda\) in the inventory model and

    • +

  • an initial condition \(X_0 \sim \psi_0\), where \(\psi_0\) is an arbitrary +distribution on \(S\).

    • +
+

The state \(S\) is set to \(\{0, \ldots, b\}\) and the matrix \(K\) is defined by +(19).

+

Now we run time forward.

+

We are interesting in computing the flow of distributions \(t \mapsto \psi_t\), +where \(\psi_t\) is the distribution of \(X_t\).

+

According to the theory developed above, we have two options:

+

Option 1 is to use simulation.

+

The first step is to simulate many independent observations the process \((X_t^m)_{m=1}^M\).

+

(Here \(m\) indicates simulation number \(m\), which you might think of as the outcome +for firm \(m\).)

+

Next, for any given \(t\), we define \(\hat \psi_t \in \dD\) as the +histogram of observations at time \(t\), or, equivalently the cross-sectional +distribution at \(t\):

+
+\[ + \hat \psi_t(x) := \frac{1}{M} \sum_{m=1}^M \mathbb 1\{X_t = x\} + \qquad (x \in S) +\]
+

Then \(\hat \psi_t(x)\) will be close to \(\PP\{X_t = x\}\) by the law of +large numbers.

+

In other words, in the limit we recover \(\psi_t\).

+

Option 2 is to insert the parameters into the right hand side of (20) +and compute \(\psi_t\) as \(\psi_0 P_t\).

+

The figure below is created using option 2, with \(\alpha = 0.6\), \(\lambda = 0.5\) and \(b=10\).

+

For the initial distribution we pick a binomial distribution.

+

Since we cannot compute the entire uncountable flow \(t \mapsto \psi_t\), we +iterate forward 200 steps at time increments \(h=0.1\).

+

In the figure, hot colors indicate initial conditions and early dates (so that the +distribution “cools” over time)

+
+
+_images/markov_prop_7_0.png +
+

Fig. 1 Probability flows for the inventory model.

+
+

In the (solved) exercises you will be asked to try to reproduce this figure.

+
+
+

Exercises

+
+

Exercise 1

+

Consider the binary (Bernoulli) distribution where outcomes \(0\) and \(1\) each have +probability \(0.5\).

+

Construct two different random variables with this distribution.

+
+
+

Exercise 2

+

Show by direct calculation that the Poisson matrices \((P_t)\) defined in +(17) satisfy the semigroup property (15).

+

Hints

+
    +

  • Recall that \(P_t(j, k) = 0\) whenever \(j > k\).

    • +

  • Consider using the binomial formula.

    • +
+
+
+

Exercise 3

+

Consider the distribution over \(S^{n+1}\) previously shown in (13), which is

+
+\[ + \mathbf P_\psi^n(x_0, x_1, \ldots, x_n) + = \psi(x_0) + P(x_0, x_1) + \times \cdots \times + P(x_{n-1}, x_n) +\]
+

Show that, for any Markov chain \((X_t)\) satisfying (10) +and \(X_0 \sim \psi\), the restriction \((X_0, \ldots, X_n)\) has joint +distribution \(\mathbf P_\psi^n\).

+
+
+

Exercise 4

+

Try to produce your own version of the figure Probability flows for the inventory model..

+

The initial condition is ψ_0 = binom.pmf(states, n, 0.25) where n = b + 1.

+
+
+
+

Solutions

+
+

Solution to Exercise 1

+

This is easy.

+

One example is to take \(U\) to be uniform on \((0, 1)\) and set \(X=0\) if \(U < +0.5\) and \(1\) otherwise.

+

Then \(X\) has the desired distribution.

+

Alternatively, we could take \(Z\) to be standard normal and set \(X=0\) if \(Z < +0\) and \(1\) otherwise.

+
+
+

Solution to Exercise 2

+

Fixing \(s, t \in \RR_+\) and \(j \leq k\), we have

+
+\[\begin{split} +\begin{aligned} + \sum_{i \geq 0} P_s(j, i) P_t(i, k) + & = + e^{-\lambda (s+t)} + \sum_{j \leq i \leq k} + \frac{ (\lambda s)^{i-j} }{(i-j)!} + \frac{ (\lambda t)^{k-i} }{(k-i)!} + \\ + & = + e^{-\lambda (s+t)} \lambda^{k-j} + \sum_{0 \leq \ell \leq k-j} + \frac{ s^\ell }{\ell!} + \frac{ t^{k-j - \ell} }{(k-j - \ell)!} + \\ + & = + e^{-\lambda (s+t)} \lambda^{k-j} + \sum_{0 \leq \ell \leq k-j} + \binom{k-j}{\ell} + \frac{s^\ell t^{k-j - \ell}}{(k-j)!} +\end{aligned} +\end{split}\]
+

Applying the binomial formula, we can write this as

+
+\[ + \sum_{i \geq 0} P_s(j, i) P_t(i, k) + = + e^{-\lambda (s+t)} + \frac{(\lambda (s + t))^{k-j}}{(k-j)!} + = P_{s+t}(j, k) +\]
+

Hence (15) holds, and the semigroup property is satisfied.

+
+
+

Solution to Exercise 3

+

Let \((X_t)\) be a Markov chain satisfying (10) and \(X_0 \sim \psi\).

+

When \(n=0\), we have \(\mathbf P_\psi^n = \mathbf P_\psi^0 = \psi\), and this +agrees with the distribution of the restriction \((X_0, \ldots, X_n) = (X_0)\).

+

Now suppose the same is true at arbitrary \(n-1\), in the sense that +the distribution of \((X_0, \ldots, X_{n-1})\) is equal to \(\mathbf P_\psi^{n-1}\) as +defined above.

+

Then

+
+\[\begin{split} + \PP \{X_0 = x_0, \ldots, X_n = x_n\} + = \PP \{X_n = x_n \,|\, X_0 = x_0, \ldots, X_{n-1} = x_{n-1} \} + \\ + \times \PP \{X_0 = x_0, \ldots, X_{n-1} = x_{n-1}\} +\end{split}\]
+

From the Markov property and the induction hypothesis, the right hand side is

+
+\[ + P (x_{n-1}, x_n ) + \mathbf P_\psi^n(x_0, x_1, \ldots, x_{n-1}) + = + P (x_n, x_{n+1} ) + \psi(x_0) + P(x_0, x_1) + \times \cdots \times + P(x_{n-1}, x_{n-1}) +\]
+

The last expression equals \(\mathbf P_\psi^n\), which concludes the proof.

+
+
+

Solution to Exercise 4

+

Here is one approach.

+

(The statements involving glue are specific to this book and can be deleted +by most readers. They store the output so it can be displayed elsewhere.)

+
+
+
α = 0.6
+λ = 0.5
+b = 10
+n = b + 1
+states = np.arange(n)
+I = np.identity(n)
+
+K = np.zeros((n, n))
+K[0, -1] = 1
+for i in range(1, n):
+    for j in range(0, i):
+        if j == 0:
+            K[i, j] = (1 - α)**(i-1)
+        else:
+            K[i, j] = α * (1 - α)**(i-j-1)
+
+
+def P_t(ψ, t):
+    return ψ @ expm(t * λ * (K - I))
+
+def plot_distribution_dynamics(ax, ψ_0, steps=200, step_size=0.1):
+    ψ = ψ_0
+    t = 0.0
+    colors = cm.jet_r(np.linspace(0.0, 1, steps))
+
+    for i in range(steps):
+        ax.bar(states, ψ, zs=t, zdir='y', 
+            color=colors[i], alpha=0.8, width=0.4)
+        ψ = P_t(ψ, t=step_size)
+        t += step_size
+
+    ax.set_xlabel('inventory')
+    ax.set_ylabel('$t$')
+
+
+ψ_0 = binom.pmf(states, n, 0.25)
+fig = plt.figure(figsize=(8, 6))
+ax = fig.add_subplot(111, projection='3d')
+plot_distribution_dynamics(ax, ψ_0)
+
+from myst_nb import glue
+glue("flow_fig", fig, display=False)
+
+plt.show()
+
+
+
+
+_images/markov_prop_7_1.png +
+
+
+
+
1
+

On a technical level, right continuity of paths for \((X_t)\) implies condition 2, as proved in Theorem 2.12 of [Lig10]. Right continuity of paths allows for jumps, but insists on only finitely many jumps in any bounded interval.

+
+
2
+

In the definition of \(P_t\) in (17), we use the convention that \(0^0 = 1\), which leads to \(P_0 = I\) and \(\lim_{t \to 0} P_t(j, k) = I(j,k)\) for all \(j,k\). These facts, along with the semigroup property, imply that \((P_t)\) is a valid Markov semigroup.

+
+
+
+
+
+ + + + +
+ +
+
+ + +
+ + Poisson Processes + Kolmogorov Backward Equation + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file diff --git a/memoryless.html b/memoryless.html new file mode 100644 index 0000000..89960a0 --- /dev/null +++ b/memoryless.html @@ -0,0 +1,771 @@ + + + + + + + + Memoryless Distributions — Continuous Time Markov Chains + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + +
+ +
+ + + + + + + + +
+ +
+ + + + + + +
+
+ +
+ + +
+
+ Contents +
+ +
+
+
+
+ +
+ +
+

Memoryless Distributions

+
+

Overview

+

Markov processes are, by definition, forgetful.

+

In particular, for any Markov processes, the distribution over future outcomes +depends only on the current state, rather than the entire history.

+

In the case of continuous time Markov chains, which jump between discrete +states, this requires that the amount of elapsed time since the last jump is +not helpful in predicting the timing of the next jump.

+

In other words, the jump times are “memoryless”.

+

It is remarkable that the only distribution on \(\RR_+\) with this +property is the exponential distribution.

+

Similarly, the only memoryless distribution on \(\ZZ_+\) is the geometric +distribution.

+

This lecture tries to clarify these ideas.

+

We will use the following imports:

+
+
+
import numpy as np
+import matplotlib.pyplot as plt
+import quantecon as qe
+from numba import njit
+from scipy.special import factorial, binom
+
+
+
+
+
+
+

The Geometric Distribution

+

Consider betting on a roulette wheel and suppose that red has come up four times in a row.

+

Since five reds in a row is an unlikely event, many people instinctively feel +that black is more likely on the fifth spin — “Surely black will come up this time!”

+

But rational thought tells us such instincts are wrong: the four previous reds make no +difference to the outcome of the next spin.

+

(Many casinos offer an unlimited supply of free alcoholic beverages in order to discourage this kind of rational analysis.)

+

A mathematical restatement of this phenomenon is: the geometric distribution is memoryless.

+
+

Memorylessness

+

Let \(X\) be a random variable supported on the nonnegative integers \(\ZZ_+\).

+

We say that \(X\) is geometrically distributed if, for some \(\theta\) satisfying \(0 \leq \theta \leq 1\),

+
+(1)\[ + \PP\{X = k\} = (1-\theta)^k \theta + \qquad (k = 0, 1, \ldots) +\]
+

An example can be constructed from the discussion of the roulette wheel above.

+

Suppose that,

+
    +

  • the outcome of each spin is either red or black,

    • +

  • spins are labeled by \(0, 1, 2, \ldots\),

    • +

  • on each spin, black occurs with probability \(\theta\) and

    • +

  • outcomes across spins are independent.

    • +
+

Then (1) is the probability that the first occurrence of black is at spin \(k\).

+

(The outcome “black” fails \(k\) times and then succeeds.)

+

Consistent with our discussion in the introduction, the geometric distribution +is memoryless.

+

For example, given any nonnegative integer \(m\), we have

+
+(2)\[ + \PP \{X = m + 1 \,|\, X > m \} = \theta +\]
+

In other words, regardless of how long we have seen only red outcomes, the +probability of black on the next spin is the same as the unconditional +probability of getting black on the very first spin.

+

To show this, we note that the left hand side is

+
+\[ + \frac{ \PP \{X = m + 1 \text{ and } X > m \} } + {\PP \{X \geq m\}} + = + \frac{ \PP \{X = m + 1 \} } + {\PP \{X > m\}} + = \frac{ (1-\theta)^{m+1} \theta } + {\sum_{k > m} (1-\theta)^k \theta } +\]
+

The right hand side simplifies to \(\theta\), completing the proof of (2).

+
+
+
+

The Exponential Distribution

+

Later, when we construct continuous time Markov chains, we will need to +specify the distribution of the holding times, which are the time intervals +between jumps.

+

As discussed above (and again below), the holding time distribution must be +memoryless, so that the chain satisfies the Markov property.

+

While the geometric distribution is memoryless, its discrete support makes it +a poor fit for the continuous time case.

+

Hence we turn to the exponential distribution, which is supported on \(\RR_+\).

+

A random variable \(Y\) on \(\RR_+\) is called exponential with rate \(\lambda\), denoted by \(Y \sim \Exp(\lambda)\), if

+
+\[ + \PP\{Y > y\} = e^{-\lambda y} + \qquad (y \geq 0) +\]
+
+

From Geometric to Exponential

+

The exponential distribution can be regarded as the “limit” of the geometric +distribution.

+

To illustrate, let us suppose that

+
    +

  • customers enter a shop at discrete times \(t_0, t_1, \ldots\)

    • +

  • these times are evenly spaced, so that \(h = t_{i+1} - t_i\) for some \(h > 0\) and all \(i \in \ZZ_+\)

    • +

  • at each \(t_i\), either zero or one customers enter (no more because \(h\) is small)

    • +

  • entry at each \(t_i\) occurs with probability \(\lambda h\) and is independent over \(i\).

    • +
+

The fact that the entry probability is proportional to \(h\) is important in +what follows.

+

You can imagine many customers passing by the shop, each entering +independently.

+

If we halve the time interval, then we also halve the probability that a +customer enters.

+

Let

+
    +

  • \(Y\) be the time of the first arrival at the shop,

    • +

  • \(t\) be a given positive number and

    • +

  • \(i(h)\) be the largest integer such that \(t_{i(h)} \leq t\).

    • +
+

Note that, as \(h \to 0\), the grid becomes finer and \(t_{i(h)} = i(h) h \to t\).

+

Writing \(i(h)\) as \(i\) and using the geometric distribution, the probability that +the first arrival occurs after \(t_{i}\) is \((1-\lambda h)^{i}\).

+

Hence

+
+\[ + \PP\{Y > t_{i} \} + = (1-\lambda h)^i + = \left( 1- \frac{\lambda i h}{i} \right)^i +\]
+

Using the fact that \(e^x = \lim_{i \to \infty}(1 + x/i)^i\) for all \(x\) and \(i +h = t_i \to t\), we obtain, for large \(i\),

+
+\[ + \PP\{Y > t\} + \approx + e^{- \lambda t} +\]
+

In this sense, the exponential is the limit of the geometric distribution.

+
+
+

Memoryless Property of the Exponential Distribution

+

The exponential distribution is the only memoryless distribution supported on \(\RR_+\), as the next theorem attests.

+
+

Theorem 1 (Characterization of the Exponential Distribution)

+
+

If \(X\) is a random variable supported on \(\RR_+\), then there exists a +\(\lambda > 0\) such that \(X \sim \Exp(\lambda)\) if and only if, for all +positive \(s, t\),

+
+(3)\[ + \PP \{X > s + t \,|\, X > s \} = \PP \{X > t\} +\]
+
+
+

Proof. To see that (3) holds when \(X\) is exponential with rate \(\lambda\), +fix \(s, t > 0\) and observe that

+
+\[ + \frac{ \PP \{X > s + t \text{ and } X > s \} } + {\PP \{X > s\}} + = + \frac{ \PP \{X > s + t \} } + {\PP \{X > s\}} + = \frac{e^{-\lambda s - \lambda t}}{e^{-\lambda s}} + = e^{-\lambda t} +\]
+

To see that the converse holds, let \(X\) be a random variable supported on \(\RR_+\) +such that (3) holds.

+

The “exceedance” function \(f(s) := \PP\{X > s\}\) then has three properties:

+
    +

  1. \(f\) is decreasing on \(\RR_+\),

    2. +

  2. \(0 < f(t) < 1\) for all \(t > 0\),

    3. +

  3. \(f(s + t) = f(s) f(t)\) for all \(s, t > 0\).

    4. +
+

The first property is common to all exceedance functions, the second is due to +the fact that \(X\) is supported on all of \(\RR_+\), and the +third is (3).

+

From these three properties we will show that

+
+(4)\[ + f(t) = f(1)^t \;\; \forall \, t \geq 0 +\]
+

This is sufficient to prove the claim because then \(\lambda := - \ln f(1)\) is a positive real number (by property 2) and, moreover,

+
+\[ + f(t) + = \exp( \ln ( f(1) ) t) + = \exp( - \lambda t) +\]
+

To see that (4) holds, fix positive integers \(m,n\).

+

We can use property 3 to obtain both

+
+\[ + f(m/n) = f(1/n)^m + \quad \text{and} \quad + f(1) = f(1/n)^n +\]
+

It follows that \(f(m/n)^n = f(1/n)^{m n} = f(1)^m\) and, raising to the power +of \(1/n\), we get (4) when \(t=m/n\).

+

The discussion so far confirms that (4) holds when \(t\) is rational.

+

So now take any \(t \geq 0\) and rational sequences \((a_n)\) and \((b_n)\) +converging to \(t\) with \(a_n \leq t \leq b_n\) for all \(n\).

+

By property 1 we have \(f(a_n) \leq f(t) \leq f(b_n)\) for all \(n\), so

+
+\[ + f(1)^{a_n} \leq f(t) \leq f(1)^{b_n} + \quad \forall \, n \in \NN +\]
+

Taking the limit in \(n\) completes the proof.

+
+
+
+

Failure of Memorylessness

+

We know from the proceeding section that any distribution on \(\RR_+\) other +than the exponential distribution fails to be memoryless.

+

Here’s an example that helps to clarify (although the support of the distribution is a proper subset of \(\RR_+\)).

+

A random variable \(Y\) has the Pareto distribution with positive parameters \(t_0, \alpha\) if

+
+\[\begin{split} + f(t) + := \PP\{Y > t\} + = + \begin{cases} + 1 & \text{ if } t \leq t_0 + \\ + (t_0 / t)^\alpha & \text{ if } t > t_0 + \end{cases} +\end{split}\]
+

As a result, with \(s > t_0\),

+
+\[ + \PP \{Y > s + t \,|\, Y > s \} + = + \frac{ \PP \{Y > s + t \} } + {\PP \{Y > s\}} + = \left( \frac{t}{t + s} \right)^\alpha +\]
+

Since this probability falls with \(s\), the distribution is not memoryless.

+

If we have waited many hours for an event (i.e., \(s\) is large), then the +probability of waiting another hour is relatively small.

+
+
+
+

Sums of Exponentials

+

A random variable \(W\) on \(\RR_+\) is said to have the Erlang +distribution if its +density has the form

+
+\[ + f(t) = \frac{\lambda^n t^{n-1}}{(n-1)!} e^{-\lambda t} + \qquad (t \geq 0) +\]
+

for some \(n \in \NN\) and \(\lambda > 0\).

+

The parameters \(n\) and \(\lambda\) are called the shape and rate +parameters respectively.

+

The next figure shows the shape for two parameterizations.

+
+
+
t_grid = np.linspace(0, 50, 100)
+
+class Erlang:
+
+    def __init__(self, λ=0.5, n=10):
+        self.λ, self.n = λ, n
+
+    def __call__(self, t):
+        n, λ = self.n, self.λ
+        return (λ**n * t**(n-1) * np.exp(-λ * t)) / factorial(n-1)
+
+e1 = Erlang(n=10, λ=0.5)
+e2 = Erlang(n=10, λ=0.75)
+
+fig, ax = plt.subplots()
+for e in e1, e2:
+    ax.plot(t_grid, e(t_grid), label=f'$n={e.n}, \lambda={e.λ}$')
+
+ax.legend()
+plt.show()
+
+
+
+
+_images/memoryless_3_0.png +
+
+

The CDF of the Erlang distribution is

+
+(5)\[ + F(t) + = \PP\{W \leq t\} + = 1 - \sum_{k=0}^{n-1} \frac{(\lambda t)^k}{k!} e^{-\lambda t} +\]
+

The Erlang distribution is of interest to us because of the following fact.

+
+

Lemma 2 (Distribution of Sum of Exponentials)

+
+

If, for some \(\lambda > 0\), the sequence \((W_i)\) is IID and exponentially +distributed with rate \(\lambda\), then \(J_n := \sum_{i=1}^n W_i\) has the Erlang +distribution with shape \(n\) and rate \(\lambda\).

+
+

This connects to Poisson process theory, as we shall soon see.

+
+
+

Exercises

+
+

Exercise 1

+

Due to its memoryless property, we can “stop” and “restart” an exponential +draw without changing its distribution.

+

To illustrate this, we can think of fixing \(\lambda > 0\), drawing from +\(\Exp(\lambda)\), and stopping and restarting whenever a threshold \(s\) is crossed.

+

In particular, consider the random variable \(X\) defined as follows:

+
    +

  • Draw \(Y\) from \(\Exp(\lambda)\).

    • +

  • If \(Y \leq s\), set \(X = Y\).

    • +

  • If not, draw \(Z\) independently from \(\Exp(\lambda)\) and set \(X = s + Z\).

    • +
+

Show that \(X \sim \Exp(\lambda)\).

+
+
+

Exercise 2

+

Fix \(\lambda = 0.5\) and \(s=1.0\).

+

Simulate 1,000 draws of \(X\) using the algorithm above.

+

Plot the fraction of the sample exceeding \(t\) for each \(t \geq 0\) (on a grid) +and compare to \(t \mapsto e^{-\lambda t}\).

+

Is the fit good? How about if the number of draws is increased?

+

Are the results in line with those of the previous exercise?

+
+
+
+

Solutions

+
+

Solution to Exercise 1

+

Let \(X\) be constructed as in the statement of the exercise and fix \(t > 0\).

+

Notice that \(X > s + t\) if and only if \(Y > s\) and \(Z > t\).

+

As a result of this fact and independence,

+
+\[ + \PP\{X > s + t\} + = \PP\{Y > s \} \PP\{Z > t\} + = e^{-\lambda(s + t)} +\]
+

At the same time,

+
+\[ + \PP\{X > s - t\} + = \PP\{Y > s - t \} + = e^{-\lambda(s - t)} +\]
+

Either way, we have \(X \sim \Exp(\lambda)\), as was to be shown.

+
+
+

Solution to Exercise 2

+

Here’s one solution, starting with 1,000 draws.

+
+
+
λ = 0.5 
+np.random.seed(1234)
+t_grid = np.linspace(0, 10, 200)
+
+@njit
+def draw_X(s=1.0, n=1_000):
+    draws = np.empty(n)
+    for i in range(n):
+        Y = np.random.exponential(scale=1/λ)
+        if Y <= s:
+            X = Y
+        else:
+            Z = np.random.exponential(scale=1/λ)
+            X = s + Z
+        draws[i] = X
+    return draws
+
+fig, ax = plt.subplots()
+draws = draw_X()
+empirical_exceedance = [np.mean(draws > t) for t in t_grid]
+ax.plot(t_grid, np.exp(- λ * t_grid), label='exponential exceedance')
+ax.plot(t_grid, empirical_exceedance, label='empirical exceedance')
+ax.legend()
+
+plt.show()
+
+
+
+
+_images/memoryless_5_0.png +
+
+

The fit is already very close, which matches with the theory in Exercise 1.

+

The two lines become indistinguishable as \(n\) is increased further.

+
+
+
fig, ax = plt.subplots()
+draws = draw_X(n=10_000)
+empirical_exceedance = [np.mean(draws > t) for t in t_grid]
+ax.plot(t_grid, np.exp(- λ * t_grid), label='exponential exceedance')
+ax.plot(t_grid, empirical_exceedance, label='empirical exceedance')
+ax.legend()
+
+plt.show()
+
+
+
+
+_images/memoryless_7_0.png +
+
+
+
+
+ + + + +
+ +
+
+ + +
+ + Continuous Time Markov Chains + Poisson Processes + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file diff --git a/objects.inv b/objects.inv new file mode 100644 index 0000000000000000000000000000000000000000..0290660886fc619e4d5fba6ff0cfd6a6fec1f998 GIT binary patch literal 897 zcmV-{1AhD?AX9K?X>NERX>N99Zgg*Qc_4OWa&u{KZXhxWBOp+6Z)#;@bUGkVd30!R zZVDqHR%LQ?X>V>iAPOTORA^-&a%F8{X>Md?av*PJAarPHb0B7EY-J#6b0A}HZE$jB zb8}^6Aa!$TZf78RY-wUH3V7O;RohM*F%W(CS6G6_rUI&7=o`0GP$8j0`x<%Iv%4m? zXKQ;2>DPDcy(8L$he(_8Ipdi#7iX9`-1xORIXe1SG&Tbi;%&f+iLJ# zk`c9ts3mtO`}` za1vpQ2)iSlM5c61F&aHD(D6_KSM~{PpisF%3Hm)N!GaTH@l(-8YbRP?7oQ~iHXKL3 zvJ#%>q4n05eH}jui6$UC*Z)`dkS`q?x&jRZr*a;UUp?1oDQ+b+t8gQnJUKf43>pj> z{T@B3Y;ZVDFd5qDsE2~G4K_YW&u^2g{Gt+6L}ETs5ksru9yO$pvnKWOawh!sApC6< zo(+Xq$*mP;hwA~oXwhe_m)LougqVfR;^^ivccDw1J6WSP zSfV3G*+%w$u1p3}g5)>EY-QtT#H9hN44ZSNC3wXf|329GxyaYC(){XolO}fRs`#g? zqIVV3o48w|n|sqhJ@hFqIje--qldIAO9+8jNBP(?yLl7or!l{*ND8>?J(}Md94(}A zIA+xO$V`^ctdyS7jK4CR^>)!8bbOSJhHgwPI;Z^c>FJj?!aXvqNmy*2>7{rB2bK3| zr)hPjHm1AooTMP02hWD(KjX|^1e%Ls_4z=%;}@MU2egY_7%MO;F}d-egDny)7UM^2 zP_`Tk;zfl>EpVI`h-_b;J<|2^QSCEKY2WipsDX+>qnT#v+6G83B&py*xV)OXYHj_d zPUm-`1V*iZn6D|^4{r>&$yX*{`Elz{B!79<-uv^SZ{G2(U>l)*Mc1FKEpDQYyKkNt Xr-84 literal 0 HcmV?d00001 diff --git a/poisson.html b/poisson.html new file mode 100644 index 0000000..8ac7e26 --- /dev/null +++ b/poisson.html @@ -0,0 +1,799 @@ + + + + + + + + Poisson Processes — Continuous Time Markov Chains + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + +
+ +
+ + + + + + + + +
+ +
+ + + + + + +
+
+ +
+ + +
+
+ Contents +
+ +
+
+
+
+ +
+ +
+

Poisson Processes

+
+

Overview

+

Counting processes count the number of “arrivals” occurring by a given time +(e.g., the number of visitors to a website, the number of customers arriving at a restaurant, etc.)

+

Counting processes become Poisson processes when the time interval between +arrivals is IID and exponentially distributed.

+

Exponential distributions and Poisson processes have deep connections to +continuous time Markov chains.

+

For example, Poisson processes are one of the simplest nontrivial examples of +a continuous time Markov chain.

+

In addition, when continuous time Markov chains jump between states, the time +between jumps is necessarily exponentially distributed.

+

In discussing Poisson processes, we will use the following imports:

+
+
+
import numpy as np
+import matplotlib.pyplot as plt
+import quantecon as qe
+from numba import njit
+from scipy.special import factorial, binom
+
+
+
+
+
+
+

Counting Processes

+

Let’s start with the general case of an arbitrary counting process.

+
+

Jumps and Counts

+

Let \((J_k)\) be an increasing sequence of nonnegative random variables +satisfying \(J_k \to \infty\) with probability one.

+

For example, \(J_k\) might be the time the \(k\)-th customer arrives at a shop.

+

Then

+
+(6)\[ + N_t := \sum_{k \geq 0} k \mathbb{1} \{ J_k \leq t < J_{k+1} \} +\]
+

is the number of customers that have visited by time \(t\).

+

The next figure illustrate the definition of \(N_t\) for a given jump sequence \(\{J_k\}\).

+
+
+
Ks = 0, 1, 2, 3
+Js = 0, 0.8, 1.8, 2.1, 3
+n = len(Ks)
+
+fig, ax = plt.subplots()
+
+ax.plot(Js[:-1], Ks, 'o')
+ax.hlines(Ks, Js[:-1], Js[1:], label='$N_t$')
+ax.vlines(Js[:-1], (0, Ks[0], Ks[1], Ks[2]), Ks, alpha=0.25)
+
+ax.set(xticks=Js[:-1],
+       xticklabels=[f'$J_{k}$' for k in range(n)],
+       yticks=(0, 1, 2, 3),
+       xlabel='$t$')
+
+ax.legend(loc='lower right')
+plt.show()
+
+
+
+
+_images/poisson_3_0.png +
+
+

An alternative but equivalent definition is

+
+\[ + N_t = \max \{k \geq 0 \,|\, J_k \leq t \} +\]
+

As a function of \(t\), the process \(N_t\) is called a counting process.

+

The jump times \((J_k)\) are sometimes called arrival times and the +intervals \(J_k - J_{k-1}\) are called wait times or holding times.

+
+
+

Exponential Holding Times

+

A Poisson process is a counting process with independent exponential holding times.

+

In particular, suppose that the arrival times are given by \(J_0 = 0\) and

+
+\[ + J_k := W_1 + \cdots W_k +\]
+

where \((W_i)\) are IID exponential with some fixed rate \(\lambda\).

+

Then the associated counting process \((N_t)\) is called a Poisson process with rate \(\lambda\).

+

The rationale behind the name is that, for each \(t > 0\), the random variable +\(N_t\) has the Poisson distribution with parameter \(t \lambda\).

+

In other words,

+
+(7)\[ + \PP\{N_t = k\} + = e^{-t \lambda} \frac{(t \lambda)^k }{k!} + \qquad (k = 0, 1, \ldots) +\]
+

For example, since \(N_t = 0\) if and only if \(W_1 > t\), we have

+
+\[ + \PP\{N_t =0\} + = \PP\{W_1 > t\} + = e^{-t \lambda} +\]
+

and the right hand side agrees with (7) when \(k=0\).

+

This sets up a proof by induction, which is time consuming but not difficult +— the details can be found in \(\S29\) of [How17].

+

Another way to show that \(N_t\) is Poisson with rate \(\lambda\) is appeal to +Lemma 2.

+

We observe that

+
+\[ + \PP\{N_t \leq n\} + = \PP\{J_{n+1} > t\} + = 1 - \PP\{J_{n+1} \leq t\} +\]
+

Inserting the expression for the Erlang CDF in (5) with shape \(n+1\) and +rate \(\lambda\), we obtain

+
+\[ + \PP\{N_t \leq n\} + = \sum_{k=0}^{n} \frac{(t \lambda )^k}{k!} e^{-t \lambda} +\]
+

This is the (integer valued) CDF for the Poisson distribution with parameter +\(t \lambda\).

+

An exercise at the end of the lecture asks you to verify that \(N(t)\) is Poisson-\((t \lambda )\) informally via simulation.

+

The next figure shows one realization of a Poisson process \((N_t)\), with jumps +at each new arrival.

+
+
+
np.random.seed(1234)
+T = 5
+Ws = np.random.exponential(size=T)
+Js = np.cumsum(Ws)
+Ys = np.arange(T)
+
+fig, ax = plt.subplots()
+
+ax.plot(np.insert(Js, 0, 0)[:-1], Ys, 'o')
+ax.hlines(Ys, np.insert(Js, 0, 0)[:-1], Js, label='$N_t$')
+ax.vlines(Js[:-1], Ys[:-1], Ys[1:], alpha=0.25)
+
+ax.set(xticks=[],
+       yticks=range(Ys.max()+1),
+       xlabel='time')
+
+ax.grid(lw=0.2)
+ax.legend(loc='lower right')
+plt.show()
+
+
+
+
+_images/poisson_5_0.png +
+
+
+
+
+

Stationary Independent Increments

+

One of the defining features of a Poisson process is that it has stationary +and independent increments.

+

This is due to the memoryless property of exponentials.

+

It means in particular that

+
    +

  1. the variables \(\{N_{t_{i+1}} - N_{t_i}\}_{i \in I}\) are independent for any +strictly increasing finite sequence \((t_i)_{i \in I}\) and

    2. +

  2. the distribution of \(N_{t+h} - N_t\) depends on \(h\) but not \(t\).

    3. +
+

A detailed proof can be found in Theorem 2.4.3 of [Nor98].

+

Instead of repeating this, we provide some intuition from a discrete +approximation.

+

In the discussion below, we use the following well known fact: If +\((\theta_n)\) is a sequence such that \(n \theta_n\) converges, then

+
+(8)\[ + \text{Binomial}(n, \theta_n) + \approx + \text{Poisson}(n \theta_n) + \quad \text{for large } n +\]
+

(The exercises ask you to examine this claim visually.)

+

We now return to the environment where we linked the +geometric distribution to the exponential.

+

That is, we fix small \(h > 0\) and let \(t_i := ih\) for all \(i \in \ZZ_+\).

+

Let \((V_i)\) be IID binary random variables with \(\PP\{V_i = 1\} = h \lambda\) for some \(\lambda > 0\).

+

Linking to our previous discussion,

+
    +

  • either one or zero customers visits a shop at each \(t_i\).

    • +

  • \(V_i = 1\) means that a customer visits at time \(t_i\).

    • +

  • Visits occur with probability \(h \lambda\), which is proportional to the +length of the interval between grid points.

    • +
+

We learned that the wait time until the first visit is +approximately exponential with rate \(t \lambda\).

+

Since \((V_i)\) is IID, the same is true for the second wait time and so on.

+

Moreover, these wait times are independent, since they depend on separate +subsets of \((V_i)\).

+

Let \(\hat N_t\) count the number of visits by time \(t\), as shown in the next figure.

+

(\(V_i = 1\) is indicated by a vertical line at \(t_i = i h\).)

+
+
+
fig, ax = plt.subplots()
+np.random.seed(1)
+T = 10
+p = 0.25
+B = np.random.uniform(size=T) < p
+N = np.cumsum(B)
+m = N[-1]  # max of N
+
+t_grid = np.arange(T)
+t_ticks = [f'$t_{i}$' for i in t_grid]
+ax.set_yticks(range(m+1))
+ax.set_xticks(t_grid)
+ax.set_xticklabels(t_ticks, fontsize=12)
+
+ax.step(t_grid, np.insert(N, 0, 0)[:-1], label='$\hat N_t$')
+
+for i in t_grid:
+    if B[i]:
+        ax.vlines((i,), (0,), (m,), ls='--', lw=0.5)
+
+ax.legend(loc='center right')
+plt.show()
+
+
+
+
+_images/poisson_7_0.png +
+
+

We expect from the discussion above that \((\hat N_t)\) approximates a Poisson process.

+

This intuition is correct because, fixing \(t\), letting \(k := \max\{i \in +\ZZ_+ \,:\, t_i \leq t\}\) and applying (8), we have

+
+\[ + \hat N_t + = \sum_{i=1}^k V_i + \sim \text{Binomial}(k, h \lambda) + \approx + \text{Poisson}(k h \lambda ) +\]
+

Using the fact that \(kh = t_k \approx t\) as \(h \to 0\), we see +that \(\hat N_t\) is approximately Poisson with rate \(t \lambda\), just as we +expected.

+

This approximate construction of a Poisson process helps illustrate the +property of stationary independent increments.

+

For example, if we fix \(s, t\), then \(\hat N_{s + t} - \hat N_s\) is the number of visits +between \(s\) and \(s+t\), so that

+
+\[ + \hat N_{s+t} - \hat N_s + = \sum_i V_i \mathbb 1\{ s \leq t_i < s + t \} +\]
+

Suppose there are \(k\) grid points between \(s\) and \(s+t\), so that \(t \approx +kh\).

+

Then

+
+\[ + \hat N_{s+t} - \hat N_s + \sim \text{Binomial}(k, h \lambda ) + \approx + \text{Poisson}(k h \lambda ) + \approx + \text{Poisson}(t\lambda) +\]
+

This illustrates the idea that, for a Poisson process \((N_t)\), we have

+
+\[ + N_{s+t} - N_s + \sim \text{Poisson}(t\lambda) +\]
+

In particular, increments are stationary (the distribution depends on \(t\) but not \(s\)).

+

The approximation also illustrates independence of increments, since, in the +approximation, increments depend on separate subsets of \((V_i)\).

+
+
+

Uniqueness

+

What other counting processes have stationary independent increments?

+

Remarkably, the answer is none:

+
+

Theorem 3 (Characterization of Poisson Processes)

+
+

If \((M_t)\) is a stochastic process supported on \(\ZZ_+\) and starting at 0 with +the property that its increments are stationary and independent, then \((M_t)\) is a Poisson process.

+
+

In particular, there exists a \(\lambda > 0\) such that

+
+\[ + M_{s + t} - M_s + \sim \text{Poisson}(t\lambda) +\]
+

for any \(s, t\).

+

The proof is similar to our earlier proof that the exponential distribution is +the only memoryless distribution.

+

Details can be found in Section 6.2 of [Par08] or +Theorem 2.4.3 of [Nor98].

+
+

The Restarting Property

+

An important consequence of stationary independent increments is the +restarting property, which means that, when simulating, we can freely stop and +restart a Poisson process at any time:

+
+

Theorem 4 (Poisson Processes can be Pause and Restarted)

+
+

If \((N_t)\) is a Poisson process, \(s > 0\) and +\((M_t)\) is defined by \(M_t = N_{s+t} - N_s\) for \(t \geq 0\), then \((M_t)\) is a +Poisson process independent of \((N_r)_{r \leq s}\).

+
+
+

Proof. Independence of \((M_t)\) and \((N_r)_{r \leq s}\) follows from indepenence of the +increments of \((N_t)\).

+

In view of the uniqueness statement above, we can verify that \((M_t)\) is a +Poisson process by showing that \((M_t)\) starts at zero, takes values in +\(\ZZ_+\) and has stationary independent increments.

+

It is clear that \((M_t)\) starts at zero and takes values in \(\ZZ_+\).

+

In addition, if we take any \(t < t'\), then

+
+\[ + M_{t'} - M_t = N_{s+t'} - N_{s + t} + \sim \text{Poisson}((t' - t) \lambda) +\]
+

Hence \((M_t)\) has stationary increments and, +using the relation \(M_{t'} - M_t = N_{s+t'} - N_{s + t}\) again, +the increments are independent as well.

+

We conclude that \((N_{s+t} - N_s)_{t \geq 0}\) is indeed a +Poisson process independent of \((N_r)_{r \leq s}\).

+
+
+
+
+

Exercises

+
+
+

Exercise 1

+

Fix \(\lambda > 0\) and draw \(\{W_i\}\) as IID exponentials with rate \(\lambda\).

+

Set \(J_n := W_1 + \cdots W_n\) with \(J_0 = 0\) and +\(N_t := \sum_{n \geq 0} n \mathbb 1\{ J_n \leq t < J_{n+1} \}\).

+

Provide a visual test of the claim that \(N_t\) is Poisson with parameter \(t +\lambda\).

+

Do this by fixing \(t = T\), generating many independent draws of \(N_T\) and +comparing the empirical distribution of the sample with a Poisson +distribution with rate \(T \lambda\).

+

Try first with \(\lambda = 0.5\) and \(T=10\).

+
+
+

Exercise 2

+

In the lecture we used the fact that \(\Binomial(n, \theta) \approx \Poisson(n \theta)\) when \(n\) is large and \(\theta\) is small.

+

Investigate this relationship by plotting the distributions side by side.

+

Experiment with different values of \(n\) and \(\theta\).

+
+
+

Solutions

+
+

Solution to Exercise 1

+

Here is one solution.

+

The figure shows that the fit is already good with a modest sample size.

+

Increasing the sample size will further improve the fit.

+
+
+
λ = 0.5
+T = 10
+
+def poisson(k, r):
+    "Poisson pmf with rate r."
+    return np.exp(-r) * (r**k) / factorial(k)
+
+@njit
+def draw_Nt(max_iter=1e5):
+    J = 0
+    n = 0
+    while n < max_iter:
+        W = np.random.exponential(scale=1/λ)
+        J += W
+        if J > T:
+            return n
+        n += 1
+
+@njit
+def draw_Nt_sample(num_draws):
+    draws = np.empty(num_draws)
+    for i in range(num_draws):
+        draws[i] = draw_Nt()
+    return draws
+
+
+sample_size = 10_000
+sample = draw_Nt_sample(sample_size)
+max_val = sample.max()
+vals = np.arange(0, max_val+1)
+
+fig, ax = plt.subplots()
+
+ax.plot(vals, [poisson(v, T * λ) for v in vals], 
+    marker='o', label='poisson')
+ax.plot(vals, [np.mean(sample==v) for v in vals], 
+    marker='o', label='empirical')
+
+ax.legend(fontsize=12)
+plt.show()
+
+
+
+
+_images/poisson_9_0.png +
+
+
+
+

Solution to Exercise 2

+

Here is one solution. It shows that the approximation is good when \(n\) is +large and \(\theta\) is small.

+
+
+
def binomial(k, n, p):
+    # Binomial(n, p) pmf evaluated at k
+    return binom(n, k) * p**k * (1-p)**(n-k)
+
+θ_vals = 0.5, 0.2, 0.1
+
+n_vals = 50, 75, 100
+
+fig, axes = plt.subplots(len(n_vals), 1, figsize=(6, 12))
+
+for n, θ, ax in zip(n_vals, θ_vals, axes.flatten()):
+
+    k_grid = np.arange(n)
+    binom_vals = [binomial(k, n, θ) for k in k_grid]
+    poisson_vals = [poisson(k, n * θ) for k in k_grid]
+    ax.plot(k_grid, binom_vals, 'o-', alpha=0.5, label='binomial')
+    ax.plot(k_grid, poisson_vals, 'o-', alpha=0.5, label='Poisson')
+    ax.set_title(f'$n={n}$ and $\\theta = {θ}$')
+    ax.legend(fontsize=12)
+
+fig.tight_layout()
+plt.show()
+
+
+
+
+_images/poisson_11_0.png +
+
+
+
+
+ + + + +
+ +
+
+ + +
+ + Memoryless Distributions + The Markov Property + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file diff --git a/prob_view.html b/prob_view.html new file mode 100644 index 0000000..e02f373 --- /dev/null +++ b/prob_view.html @@ -0,0 +1,512 @@ + + + + + + + + A Probabilistic View — Continuous Time Markov Chains + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + +
+ +
+ + + + + + + + +
+ +
+ + + + + + +
+
+ +
+ + +
+
+ Contents +
+ +
+
+
+
+ +
+ +
+

A Probabilistic View

+
+

Overview

+

Let \(S\) be a countable state space and let \(Q\) be a conservative intensity +matrix on \(S\).

+

We know that \(Q\) generates a UC Markov semigroup, and from this semigroup we +can produce a continuous time Markov chain via the associated joint distribution.

+

This method of building chains from intensity matrices is important from a +theoretical perspective but less helpful in terms of intuition and simulation.

+

In this lecture we provide another point of view.

+

In particular, we provide two different ways to build a Markov chain with +intensity matrix \(Q\).

+

One is via jump chains with state dependent jump rates.

+

The second is another probabilistic construction, sometimes called Gillespie’s +Algorithm.

+

These two constructions provide intuition from multiple perspectives and +valuable simulation algorithms.

+

We will use the following imports

+
+
+
import numpy as np
+import scipy as sp
+import matplotlib.pyplot as plt
+import quantecon as qe
+from numba import njit
+from scipy.linalg import expm
+
+
+
+
+
+
+

From Intensity Matrix to Jump Chain

+

Let us agree to call \((\lambda, K)\) a jump chain pair if \(\lambda\) is a +map from \(S\) to \(\RR_+\) and \(K\) is a Markov matrix on \(S\).

+

It is easy to verify that the matrix \(Q\) on \(S\) defined by

+
+(53)\[ + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +\]
+

is an intensity matrix.

+

(We saw in an earlier lecture that \(Q\) is the intensity +matrix for the jump chain \((X_t)\) built via Algorithm 6 from jump +chain pair \((\lambda, K)\).)

+

As we now show, every intensity matrix admits the decomposition in +(53) for some jump chain pair.

+
+

Construction

+

Given a intensity matrix \(Q\), set

+
+\[ + \lambda(x) := -Q(x, x) + \qquad (x \in S) +\]
+

Next we build \(K\), first along the principle diagonal via

+
+\[\begin{split} + K(x,x) = + \begin{cases} + 0 & \text{ if } \lambda(x) > 0 + \\ + 1 & \text{ otherwise} + \end{cases} +\end{split}\]
+

Thus, if the rate of leaving \(x\) is positive, we set \(K(x,x) = 0\), so that the +embedded jump chain moves away from \(x\) with probability one when the next +jump occurs.

+

Otherwise, when \(Q(x,x) = 0\), we stay at \(x\) forever, so \(x\) is an absorbing +state.

+

Off the principle diagonal, where \(x \not= y\), we set

+
+\[\begin{split} + K(x,y) = + \begin{cases} + \frac{Q(x,y)}{\lambda(x)} & \text{ if } \lambda(x) > 0 + \\ + 0 & \text{ otherwise } + \end{cases} +\end{split}\]
+

The exercises below ask you to confirm that, for \(\lambda\) and \(K\) just defined,

+
    +

  1. \((\lambda, K)\) is a jump chain pair and

    2. +

  2. the intensity matrix \(Q\) satisfies (53).

    3. +
+

We call \((\lambda, K)\) the jump chain decomposition of \(Q\).

+

We summarize in a lemma.

+
+

Lemma 21

+
+

A matrix \(Q\) on \(S\) is an intensity matrix if and only if there exists a jump +chain pair \((\lambda, K)\) such that (53) holds.

+
+
+
+

The Conservative Case

+

We know from Example 20 that an intensity matrix \(Q\) is conservative +if and only if \(\lambda\) is bounded.

+

Moreover, we saw in Theorem 17 that the pairing between conservative +intensity matrices and UC Markov semigroups is one-to-one.

+

This leads to the following result.

+
+

Theorem 22

+
+

On \(S\), there exists a one-to-one correspondence between the following sets of +objects:

+
    +

  1. The set of all jump chain pairs \((\lambda, K)\) such that \(\lambda\) is bounded.

    2. +

  2. The set of all conservative intensity matrices.

    3. +

  3. The set of all UC Markov semigroups.

    4. +
+
+
+
+

Simulation

+

In view of the preceding discussion, we have a simple way to simulate a Markov +chain given any conservative intensity matrix \(Q\).

+

The steps are

+
    +

  1. Decompose \(Q\) into a jump chain pair \((\lambda, K)\).

    2. +

  2. Simulate via Algorithm 6.

    3. +
+

Recalling our discussion of the Kolmogorov backward equation, we know that +this produces a Markov chain with Markov semigroup +\((P_t)\) where \(P_t = e^{tQ}\) for \(Q\) satisfying (53).

+

(Although our argument assumed finite \(S\), the proof goes through when +\(S\) is contably infinite and \(Q\) is conservative with very minor changes.)

+

In particular, \((X_t)\) is a continuous time Markov chain with intensity matrix +\(Q\).

+
+
+
+

The Gillespie Algorithm

+

Exponential clocks.

+

Show by simulation that distributions coincide with \(\psi_0 P_t\).

+
+
+

Exercises

+
+

Exercise 1

+

Let \(Q\) be any intensity matrix on \(S\).

+

Prove that the jump chain decomposition of \(Q\) is in fact a jump chain pair.

+

Prove that, in addition, this decomposition \((\lambda, K)\) satisfies (53).

+
+
+
+

Solutions

+
+

Solution to Exercise 1

+

Let \(Q\) be an intensity matrix and let \((\lambda, K)\) be the jump chain +decomposition of \(Q\).

+

Nonnegativity of \(\lambda\) is immediate from the definition of an intensity +matrix.

+

To see that \(K\) is a Markov matrix we fix \(x \in S\) and suppose first that +\(\lambda(x) > 0\).

+

Then

+
+\[ + \sum_y K(x, y) + = \sum_{y \not= x} K(x,y) + = \sum_{y \not= x} \frac{Q(x,y)}{\lambda(x)} + = 1 +\]
+

If, on the other hand, \(\lambda(x) = 0\), then +\(\sum_y K(x, y) = 1\), is immediate from the definition.

+

As \(K\) is nonnegative, we see that \(K\) is a Markov matrix.

+

Thus, \((\lambda, K)\) is a valid jump chain pair.

+

The proof that \(Q\) and \((\lambda, K)\) satisfy (53) is mechanical and +the details are omitted.

+

(Try working case-by-case, with \(\lambda(x) = 0, x=y\), \(\lambda(x) > 0, x=y\), etc.)

+
+
+
+ + + + +
+ +
+
+ + +
+ + UC Markov Semigroups + Stationarity and Ergodicity + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file diff --git a/proof-proof.html b/proof-proof.html new file mode 100644 index 0000000..2759890 --- /dev/null +++ b/proof-proof.html @@ -0,0 +1,436 @@ + + + + + + + + Proof Index — Continuous Time Markov Chains + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + + + +
+ + +
+ +
+
+
+
+ +
+ + +

Proof Index

+ +
+ algorithm-0 | + diffexpmap | + ecuc | + ejc_algo | + equivirr | + erlexp | + example-0 | + example-1 | + exp_unique | + imatjc | + intvsmk | + intvsmk_c | + jccs | + jctosg | + lemma-1 | + scintcon | + statfromq | + theorem-0 | + theorem-1 | + theorem-2 | + ucsgec | + usmg +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
 
+ algorithm-0
+ algorithm-0 (markov_prop) + algorithm
 
+ diffexpmap
+ diffexpmap (generators) + lemma
 
+ ecuc
+ ecuc (generators) + example
 
+ ejc_algo
+ ejc_algo (kolmogorov_bwd) + algorithm
 
+ equivirr
+ equivirr (ergodicity) + lemma
 
+ erlexp
+ erlexp (memoryless) + lemma
 
+ example-0
+ example-0 (generators) + example
 
+ example-1
+ example-1 (uc_mc_semigroups) + example
 
+ exp_unique
+ exp_unique (memoryless) + theorem
 
+ imatjc
+ imatjc (prob_view) + lemma
 
+ intvsmk
+ intvsmk (kolmogorov_fwd) + theorem
 
+ intvsmk_c
+ intvsmk_c (kolmogorov_fwd) + corollary
 
+ jccs
+ jccs (uc_mc_semigroups) + example
 
+ jctosg
+ jctosg (kolmogorov_bwd) + lemma
 
+ lemma-1
+ lemma-1 (kolmogorov_bwd) + lemma
 
+ scintcon
+ scintcon (uc_mc_semigroups) + lemma
 
+ statfromq
+ statfromq (ergodicity) + theorem
 
+ theorem-0
+ theorem-0 (poisson) + theorem
 
+ theorem-1
+ theorem-1 (poisson) + theorem
 
+ theorem-2
+ theorem-2 (ergodicity) + theorem
 
+ ucsgec
+ ucsgec (generators) + theorem
 
+ usmg
+ usmg (uc_mc_semigroups) + theorem
+ + +
+ +
+
+ + +
+ + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file diff --git a/search.html b/search.html new file mode 100644 index 0000000..b906232 --- /dev/null +++ b/search.html @@ -0,0 +1,250 @@ + + + + + + + + Search — Continuous Time Markov Chains + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + + + +
+ + +
+ +
+
+
+
+ +
+ +

Search

+
+ +

+ Please activate JavaScript to enable the search + functionality. +

+
+

+ From here you can search these documents. Enter your search + words into the box below and click "search". Note that the search + function will automatically search for all of the words. Pages + containing fewer words won't appear in the result list. +

+
+ + + +
+ +
+ +
+ +
+ +
+
+ + +
+ + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file diff --git a/searchindex.js b/searchindex.js new file mode 100644 index 0000000..b25b7a8 --- /dev/null +++ b/searchindex.js @@ -0,0 +1 @@ +Search.setIndex({docnames:["ergodicity","generators","intro","kolmogorov_bwd","kolmogorov_fwd","markov_prop","memoryless","poisson","prob_view","uc_mc_semigroups","zreferences"],envversion:{"sphinx.domains.c":1,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":1,"sphinx.domains.index":1,"sphinx.domains.javascript":1,"sphinx.domains.math":2,"sphinx.domains.python":1,"sphinx.domains.rst":1,"sphinx.domains.std":1,"sphinx.ext.intersphinx":1,sphinx:56},filenames:["ergodicity.md","generators.md","intro.md","kolmogorov_bwd.md","kolmogorov_fwd.md","markov_prop.md","memoryless.md","poisson.md","prob_view.md","uc_mc_semigroups.md","zreferences.md"],objects:{},objnames:{},objtypes:{},terms:{"100_000":3,"10_000":[6,7],"1_000":6,"1e5":7,"\u03b1":[3,5],"\u03b3":3,"\u03b8":7,"\u03b8_val":7,"\u03bb":[3,5,6,7],"\u03c8":[3,4,5],"\u03c8_0":[3,5],"\u03c8_00":4,"\u03c8_01":4,"\u03c8_02":4,"\u03c8_t":3,"abstract":[1,4,5],"break":5,"case":[0,1,3,6,7],"class":[6,9],"final":[3,5],"function":[3,4,5,6,7,9,10],"import":[0,1,3,4,5,6,7,8],"long":[5,6],"new":[3,5,7],"return":[1,3,4,5,6,7],"switch":5,"true":[0,1,3,4,5,7],"try":[3,4,5,7,8,9],"while":[1,3,4,5,6,7,9],And:4,Are:6,But:[4,6,9],Doing:[],For:[0,1,3,4,5,6,7,9],Its:5,M_s:7,N_s:[5,7],Not:5,ODE:[0,1,4],ODEs:[1,3],One:[1,3,5,7,8],P_s:[3,5],Such:1,That:[5,7],The:0,Then:[5,6,7,8],There:[0,1,4,5],These:[4,5,8],U_s:1,Use:0,Uses:2,Using:[1,4,6,7],V_s:3,With:5,X_s:5,__call__:6,__init__:6,_axinfo:4,a_n:6,abbrevi:[1,5],abl:1,about:[1,3,5,6],abov:[1,3,4,5,6,7,9],absorb:8,access:0,accord:[4,5],across:6,actual:3,adam:10,adapt:4,add:5,add_collection3d:4,add_subplot:[4,5],added:[0,2],addit:[1,5,7,8,9],admit:8,adopt:[1,3],affirm:5,after:[1,3,4,5,6],again:[1,4,5,6,7,9],agre:[5,7,8],aim:[1,5,9],alcohol:6,algebra:[1,5],algorithm:[4,6,10],align:[3,5],all:[0,1,3,4,5,6,7,8,9],allow:[1,3,5],along:[3,4,5,8,9],alpha:[1,3,4,5,6,7],alreadi:[4,5,6,7],also:[1,3,4,5,6,7,9],altern:[0,5,7,9],although:[4,5,6,8],alwai:[1,3,4,5,9],american:10,amount:[3,5,6],analog:[0,1,3,4,5,9],analysi:[6,10],analyt:[3,4],analyz:5,angl:4,ani:[0,1,3,4,5,6,7,8,9],anoth:[1,3,5,6,7,8,9],answer:[3,5,7],aperiod:0,appeal:[7,9],appear:[],append:[4,5],applebaum2019semigroup:1,appli:[3,4,5,7,9],appliat:9,applic:[1,2,4,5,9,10],approach:[3,4,5,10],appropri:1,approx:[6,7],approxim:[3,7],arang:[3,5,7],arbitrari:[1,3,5,7,9],argu:4,argument:[1,3,4,5,8,9],aris:[1,9],arrai:[4,5],arriv:[3,5,6,7],art3d:[0,4],ask:[3,5,7,8,9],assert:4,associ:[0,1,3,4,7,8,9],assum:[0,1,3,4,5,8,9],assumpt:[3,4],attent:9,attest:6,author:[],autom:[],automat:5,autonom:4,avoid:[1,4],awai:8,axes3d:[0,4,5],axes:7,axi:4,b_n:6,back:[4,5],backward:[1,8,9],banach:[1,9],bar:[0,3,5],basic:1,becaus:[0,1,3,4,5,6,7,9],becom:[3,5,6,7],been:[4,5],befor:[4,5],begin:[3,4,5,6,8,9],behav:[],behind:7,being:[1,4],below:[1,3,4,5,6,7,8,9],benefici:[],bernoulli:5,bet:6,beta:1,better:1,between:[1,3,4,5,6,7,8,9],beverag:6,big:1,binari:[5,7],binom:[3,5,6,7],binom_v:7,binomi:[3,5,7],black:6,bob05:[1,10],bobrowski:10,book:5,both:[4,5,6,9],bound:[1,5,8],bounded:[1,9],brief:9,brownian:10,build:[5,8],built:8,busi:10,c_0:[1,9],calcul:[3,5],calculu:10,call:[0,1,3,4,5,6,7,8,9],cambridg:10,can:[0,1,3,4,5,6,8,9],cannot:[1,5],care:[1,4],carri:[4,5],cartiesian:5,casino:6,cauchi:1,cdf:[6,7],cdot:[0,1,3,4,5,7,9],center:7,certain:4,certainli:1,chain:[1,6,7,9,10],challeng:1,chang:[1,3,4,5,6,8],chapman:5,chapter:1,character:0,charcter:9,check:[1,4,5,9],choic:9,choos:9,chosen:[4,5],circumv:1,claim:[1,3,6,7,9],clarifi:[1,3,4,5,6],clear:[3,7],clearer:5,clearli:9,clock:8,close:[5,6],code:[3,4],coeffici:3,coincid:[1,8],collect:[5,9],colon:[3,9],color:[4,5],column:4,combin:[1,3,4,9],come:[1,6],common:[5,6],commonli:4,compar:[6,7],comparison:4,complet:[1,5,6,9],complex:3,complic:1,composit:[1,9],comput:[5,10],concern:1,conclud:[4,5,7,9],concret:5,condit:[0,3,4,5],confirm:[3,4,6,8,9],confus:5,connect:[1,4,5,6,7,9],consequ:7,conserv:[0,9],consid:[0,1,4,5,6,9],consist:[3,5,6,9],constant:[1,3,4,9],construct:[3,4,6,7],consum:7,contabl:8,contain:9,context:4,continu:[3,6,7,8,9,10],contradict:9,conveni:[3,5],convent:5,converg:[1,6,7,9],convergence_plot:4,convers:[0,4,6,9],convert:3,convinc:3,cool:[4,5],corollari:5,correct:[4,5,7],correspond:[4,5,8,9],could:5,countabl:[0,4,8,9],cours:5,cover:[5,9],crc:10,creat:[4,5],cross:[5,6],cumsum:7,current:[4,5,6,9],curv:[],custom:[3,5,6,7],d_t:0,daniel:10,darkblu:4,data:5,date:[4,5],decompos:[8,9],decomposit:8,decreas:6,deduc:9,deep:7,def:[3,4,5,6,7],defer:4,defin:[1,3,4,5,6,7,8,9],definit:[0,1,3,4,5,6,7,8,9],delai:5,delet:5,denot:[1,3,4,5,6],densiti:6,depend:[4,5,6,7,8],depthshad:4,deriv:[1,3,4,9],describ:[3,4,5],descript:[1,3,4,5],desir:5,despit:1,detail:[1,5,7,8,9],determin:[5,9],determinist:3,develop:5,diagon:[4,8],differ:[0,3,5,6,7,8],differenti:[4,9],diffexpmap:[],difficult:[3,5,7],diffus:[1,4],dimension:[1,3,4,5,9],direct:[1,5],directli:[1,4,9],discourag:6,discov:[],discret:[1,6,7,9],discuss:[1,3,4,5,6,7,8,9],displai:[0,5],distinct:0,distract:4,distribut:[1,3,7,8,9],doe:[1,5],domain:1,domin:9,done:[1,3],dougla:10,down:[3,4,5],downarrow:[1,4],downsid:[],draw:[3,5,6,7],draw_nt:7,draw_nt_sampl:7,draw_x:[3,6],drawn:3,driven:1,dtype:3,due:[3,5,6,7],dynam:4,each:[1,3,4,5,6,7,9],earli:[4,5],earlier:[3,7,8,9],easi:[0,1,3,4,5,8],easier:[0,3,9],easili:[1,3,4,9],econom:5,economist:5,ecuc:[],effici:5,either:[5,6,7],elaps:6,eleg:5,element:[0,1,3,4,5,9,10],ell:5,ell_1:[0,1,9],els:[3,5,6],elsewher:5,embed:[8,9],emphasi:4,empir:[6,7],empirical_exceed:6,emploi:4,empti:[3,6,7],empty_lik:3,encourag:3,end:[0,3,4,5,6,7,8,9],enjoi:1,enough:[5,9],ensur:9,enter:6,entir:[1,3,5,6],entri:[3,6],environ:7,equal:[1,3,4,5,9],equat:[1,5,8,9,10],equival:[0,1,4,5,7,9],erlang:[6,7],essenc:[],essenti:9,establish:[0,1,9],etc:[7,8],etienn:10,evalu:[3,5,7],even:[1,5],evenli:6,event:6,everi:[0,1,4,5,8,9],everywher:[1,9],evolut:[1,4,5],evolv:[4,5],exact:[4,9],exactli:5,examin:[4,7],exampl:[1,3,4,6,7,8,9],exceed:6,excel:1,exercis:[],exist:[0,1,4,5,6,7,8,9],exp:[3,5,6,7],exp_uniqu:[],expect:[3,4,5,7,9],experi:[3,7],explicitli:[3,4,5],explod:[1,9],explor:[1,5],explos:9,expm:[3,4,5,8],exponenti:[4,5,8,9],exposit:5,express:[1,5,7,9],extend:4,extens:[1,5],fact:[1,3,4,5,6,7,8,9],factori:[6,7],fail:[1,3,5,6],fall:[5,6,9],fals:[4,5],famili:[5,9],familiar:[1,5],far:[5,6],featur:7,feel:6,ff_:5,field:[],fifth:6,fig:[3,4,5,6,7],figsiz:[4,5,7],figur:[0,3,4,5,6,7],financ:10,finer:6,finit:[0,1,3,4,7,8],firm:5,first:[0,1,3,4,5,6,7,8,9],fit:[1,6,7],five:6,fix:[1,3,4,5,6,7,8],flatten:7,flow:[0,1,4,9],flow_fig:5,flow_plot:4,fncoex:[],fnpde:[],focu:[1,3,9],focus:5,foguel:0,fokker:4,follow:[0,1,3,4,5,6,7,8,9],fontsiz:[4,7],foobar:[],foral:[1,6],forev:8,forget:6,form:[1,3,5,6,9],formal:5,formula:5,fortun:[5,9],forward:[1,5,9],found:[1,4,7],four:6,frac:[0,1,3,4,5,6,7,8,9],fraction:6,framework:1,fran:10,free:6,freeli:7,from:[0,1,5,7,9],full:[1,3,5],fulli:1,fundament:[4,5],further:[3,5,6,7],futur:[5,6],gall:10,gamma:3,gener:[0,3,4,5,7,8,9],genom:10,geometr:[3,5,7],geq:[0,1,3,4,5,6,7,9],get:[0,4,5,6],give:[1,3,4,5,9],given:[0,1,3,4,5,6,7,8,9],glue:5,goe:[5,8],good:[6,7],govern:5,great:3,grid:[4,5,6,7],grow:9,guess:3,h_n:1,halv:6,hand:[1,3,5,6,7,8,9],happen:[],hard:[1,3,9],harder:3,has:[0,1,3,4,5,6,7,9],hat:[3,4,5,7,9],have:[0,1,3,4,5,6,7,8,9],haven:4,help:[3,5,6,7,8],henc:[0,1,3,4,5,6,7,9],henceforth:[4,5],here:[0,3,4,5,6,7,9],high:1,highli:[],hill:9,hint:[1,4,5],histogram:[3,5],histori:[5,6],hline:7,hold:[0,1,3,4,5,6,8,9],homogen:[4,5],hope:9,hot:[4,5],hour:6,how17:[7,10],how:[1,4,5,6,9],howard:10,howev:[1,9],hypothesi:5,idea:[0,1,3,4,5,6,7],ideal:1,ident:[1,3,5,9],identif:9,identifi:[0,1,4,5,9],iff:3,iid:[3,5,6,7],illustr:[5,6,7],imagin:6,immedi:[3,4,5,8,9],impli:[0,1,3,5,9],implic:4,implicitli:3,impos:4,improv:7,includ:[0,5],increas:[6,7],increment:5,inde:[1,3,5,7],indepen:7,independ:[3,5,6],independent_draw:3,index:5,indic:[1,4,5,7],indirectli:[],indistinguish:6,induc:9,induct:[5,7],inequ:[0,9],infin:9,infinit:[1,4,5,8,9],infinitesim:[1,3,4],inflow:4,inform:[3,5,7],infti:[1,3,4,5,6,7,9],initi:[1,3,4,5,9],input:[3,5],insert:[5,7,9],insight:3,insist:5,instead:[3,7],instinct:6,int64:3,int_0:3,integ:[5,6,7],intens:[0,1,4,5],interest:[1,5,6],interpret:[1,5],interv:[5,6,7],introduc:[1,5,9],introduct:[1,5,6,10],introductori:2,intuit:[3,4,5,7,8,9],intvsmk:[],invari:4,inventori:4,invert:1,investig:[3,7],involv:5,isaac:3,issu:[1,9],iter:5,its:[0,1,5,6,7],itself:[1,4,5,9],j_0:[3,5,7],j_1:3,j_k:[3,5,7],j_n:[6,7],j_val:5,jame:10,jctosg:[],jean:10,jet_r:[4,5],john:10,joint:[4,8],juggl:4,jump:[6,9],just:[0,4,5,7,8,9],justifi:5,k_grid:7,kannappan:10,kbe:[],kei:4,kfe:[],kind:6,know:[5,6,8,9,10],known:[5,7],kolmogorov:[1,5,8,9],label:[5,6,7],lack:[3,9],lambda:[3,4,5,6,7,8,9],larg:[3,5,6,7],largest:6,last:[0,1,3,5,6,9],later:[3,4,5,6],law:5,ldot:[5,6,7],lead:[3,4,5,8,9],learn:[4,5,7],least:[0,4,9],leav:[4,8],lectur:[1,3,4,5,6,7,8,9],left:[3,4,6,9],legend:[5,6,7],lemma:[1,4,7,8,9],len:7,length:[1,5,7],leq:[0,1,3,5,6,7,9],less:[4,8],let:[0,1,3,4,5,6,7,8,9],level:[1,4,5],lg16:[5,10],lig10:[5,10],liggett:10,light:9,like:[1,4,5,6],likewis:9,lim_:[1,4,5,6,9],liminf_:0,limit:[1,4,5,6],linalg:[3,4,5,8],line:[6,7],linear:[3,9],link:[0,7,9],linop:1,linopel:9,linspac:[4,5,6],list:[2,3],littl:5,loc:7,locat:1,look:[3,5,9],lose:3,low:3,lower:7,m_t:7,made:[1,3],main:[3,4,9],major:1,majortick:4,make:[0,1,5,6],mani:[0,1,3,5,6,7],map:[1,3,4,5,8,9],mapsto:[1,3,4,5,6,9],marker:7,markov:[0,1,3,4,6,7,8,10],markov_prop:0,markovian:5,martingal:10,mass:3,match:6,materi:1,mathbb:[3,4,5,7,9],mathbf:5,mathemat:[3,5,6,10],matplotlib:[0,3,4,5,6,7,8],matric:[1,4,5,8],matrici:9,matrix:[0,1,3,5,9],max:7,max_it:[3,7],max_val:7,max_x:[3,4],mean:[3,5,6,7],measur:5,mechan:8,media:10,memoryless:[1,5,7],mental:5,method:8,might:[3,5,7,9],mild:[4,5,9],milton:10,min:[3,5],minor:[3,8],minu:4,model:4,modest:7,modif:4,modifi:3,more:[1,3,4,5,6,9],moreov:[1,4,5,6,7,8,9],most:[0,1,4,5],motion:10,move:[4,5,8],mpl_toolkit:[0,4,5],mplot3d:[0,4,5],mtk:0,multipl:[1,5,8,9],must:[0,3,4,5,6],myst_nb:5,n_r:[5,7],n_t:[5,7,9],n_val:7,name:[3,7],natur:[1,4,5],necessarili:7,need:[1,3,4,5,6,9],network:10,newton:3,next:[0,1,3,5,6,7,8,9],njit:[0,3,4,5,6,7,8],none:7,nonexplos:9,nonneg:[3,4,5,6,7,8],nontrivi:[4,7],nor98:[7,9,10],norm:[0,1,9],normal:5,norri:10,notat:[1,5],note:[3,5,6,9],noth:5,notic:[3,6],notion:[5,9],now:[0,3,4,5,6,7,8,9],num_draw:[3,7],numba:[0,3,4,5,6,7,8],number:[1,5,6,7,10],numpi:[0,3,4,5,6,7,8],obei:3,object:[4,5,8],observ:[1,3,4,5,6,7],obtain:[3,4,5,6,7,9],obviou:[5,9],obvious:[3,5],occur:[5,6,7,8,9],occurr:6,odd:10,off:[4,8],offer:6,often:[1,3,4,5],ois:10,omega:1,omit:[8,9],onc:1,one:[0,1,3,4,5,6,7,8,9],ones:[1,3,4],onli:[0,4,5,6,7,8,9],oper:[0,4,9],option:5,order:[4,5,6],ordinari:[4,5],origin:4,orthogon:[],other:[1,3,4,5,6,7,8,9],otherwis:[0,3,5,8,9],our:[1,3,4,5,6,7,8,9],out:[0,1,4,5,9],outcom:[5,6],outflow:4,output:[3,5],outsid:9,over:[1,4,5,6],own:5,p_0:[1,3,4,5,9],p_h:[1,4],p_t:[0,1,3,4,5,8,9],pair:[1,4,5,8],palaniappan:10,par08:[7,10],paramet:[3,5,6,7],parameter:[4,6],pardoux:10,pareto:[5,6],parsimoni:5,part:3,particular:[1,3,4,6,7,8,9],pass:6,past:5,path:[1,3,4,5,9],pde:[1,3],peopl:6,perspect:[4,8],phenomenon:6,phi:[5,9],pi_t:5,pick:[0,1,5,9],pin:4,place:[1,5],plan:3,planck:4,plot:[5,6,7],plot_distribution_dynam:5,plt:[0,3,4,5,6,7,8],plu:5,plug:5,pmatrix:[4,9],pmf:[3,5,7],point:[0,1,5,7,8],pointwis:[1,4,9],poisson:[1,6,9],poisson_v:7,poly3dcollect:[0,4],poor:6,posit:[0,3,5,6,8],possess:5,possibl:0,postmultipli:4,power:[1,5,6,9],practic:5,prasanna:10,preced:[5,8],precis:[1,5],predict:[5,6],premultipli:[4,9],present:3,press:10,previou:[1,3,5,6,7,9],previous:[4,5,9],primit:[3,5],principl:8,prior:5,probabilist:[3,9],probabl:[0,3,4,5,6,7,8,9,10],problem:[1,4,5,9],problemat:1,procedur:[],proceed:6,process:[1,3,6,9,10],prod_:[],produc:[5,8],product:[0,1,3,5,9],project:[4,5],proof:[0,1,3,4,5,6,7,8,9],proper:6,properli:1,properti:[1,4,9],proport:[6,7],prove:[0,1,3,4,5,6,8,9],provid:[1,4,5,7,8],psi:[0,3,4,5],psi_0:[4,5,8],psi_:4,psi_t:[3,4,5],purchas:5,pure:3,push:4,pyplot:[0,3,4,5,6,7,8],qquad:[3,4,5,6,7,8,9],quad:[1,3,4,6,7],quantecon:[0,3,4,5,6,7,8],question:[3,4,5],queue:1,quickli:9,quit:[1,5],rais:6,random:[3,5,6,7],rang:[3,4,5,6,7],rate:[3,4,6,7,8,9],rather:[1,3,5,6],ration:6,rational:7,rcS:5,reach:3,reader:[1,5],real:6,realiz:7,realli:4,rearrang:3,reason:3,reassur:3,recal:[1,3,4,5,8,9],recov:[1,5],red:6,reduc:5,refer:[],regard:[3,4,5,6],regardless:6,regular:[],rel:[1,6,9],relat:[4,7,9],relationship:7,relax:[3,5],remain:4,remark:[6,7],repeat:[3,4,7,9],rephras:4,replac:5,report:5,repres:[1,4,5,9],represent:[1,3],reproduc:5,requir:[1,4,6,9],respect:[3,5,6],rest:4,restart:[5,6],restat:6,restaur:7,restrict:[1,3,4,9],result:[0,1,3,4,5,6,8,9],revers:1,review:5,rewrit:3,right:[0,1,3,4,5,6,7,9],roulett:6,row:[3,4,5,6,9],rr_:[1,3,5,6,8,9],rule:4,run:[5,9],s29:7,sahoo:10,sai:[0,1,4,5,6,9],said:[5,6],same:[1,3,4,5,6,7,9],sampl:[6,7],sample_s:7,satisfi:[1,3,4,5,6,7,8,9],saw:[1,8,9],scalar:[1,3,9],scale:[3,5,6,7],scatter:4,scenario:9,scienc:10,scientist:3,scintcon:[],scipi:[0,3,4,5,6,7,8],searchsort:5,second:[1,3,4,6,7,8,9],section:[1,4,5,6,7,9],see:[0,1,3,4,5,6,7,8,9],seed:[5,6,7],seek:0,seem:[3,5],seen:[1,3,4,6],select:[],self:6,semigroup:[0,4,8],send:4,sens:[1,4,5,6],separ:7,sequenc:[0,1,3,5,6,7],set:[0,1,3,4,6,7,8,9],set_facecolor:4,set_pad:4,set_titl:7,set_xlabel:5,set_xtick:7,set_xticklabel:[4,7],set_ylabel:5,set_ytick:7,set_yticklabel:4,set_zticklabel:4,shall:[1,6],shape:[6,7],shift:5,shine:3,shop:[6,7],shorter:0,shortest:0,should:1,show:[1,3,4,5,6,7,8,9],shown:[0,5,6,7],side:[3,5,6,7,9],sigma:5,sim:[5,6,7],sim_path:5,similar:[1,5,7],similarli:6,simpl:[1,5,8,9],simplest:[5,7],simplex:4,simpli:5,simplifi:[3,5,6],simul:[3,4,6,7],sinc:[0,1,3,5,6,7,9],situat:[],size:[5,7],sk11:[1,10],slightli:1,small:[0,3,5,6,7],soc:10,solut:[],solv:[1,3,4,5,9],some:[0,1,3,4,5,6,7,8,9],someth:1,sometim:[4,7,8],son:10,soon:[1,5,6],sound:3,space:[0,3,6,8,9],special:[1,3,5,6,7,9],specif:[1,5],specifi:[1,6],spin:6,springer:10,squar:4,stage:5,stai:8,standard:5,start:[0,1,3,4,5,6,7],stat:[3,5],state:[0,1,4,6,7,8],statement:[0,1,4,5,6,7,9],stationar:5,stationari:5,steinhau:1,step:[0,3,5,7,8],step_siz:5,still:[3,9],stochast:[5,7,10],stock:5,stop:[6,7],store:5,str13:[3,10],strictli:[0,7],strong:5,strongli:4,stroock:10,structur:5,studi:[1,3],submultipl:1,subplot:[3,5,6,7],subset:[6,7],succe:6,suffic:1,suffici:[0,6],suggest:[0,1,4],suitabl:4,sum:[0,1,3,4,5,9],sum_:[0,1,3,4,5,6,7,8,9],sum_i:[3,5,7,8,9],sum_n:9,sum_x:[5,9],sum_z:[3,5,9],summabl:9,summar:[5,8],summari:5,sup_:[1,5],sup_x:[5,9],suppli:[3,6,9],support:[5,6,7],suppos:[0,3,4,5,6,7,8,9],supremum:5,sure:[6,9],swap:5,systemat:4,t_0:6,t_1:[5,6],t_grid:[6,7],t_i:[5,6,7],t_k:[5,7],t_n:1,t_tick:7,take:[0,1,3,4,5,6,7,9],taken:[3,4,5],tan:9,tau:[1,3,5],technic:[1,3,4,5],tell:[1,4,6],term:[0,3,8],terminolog:[1,4,5],test:7,text:[0,1,2,3,4,5,6,7,8,9],than:[1,3,4,5,6],thei:[4,5,7,9],them:[3,5],theorem:[1,4,5,6,7,8,9],theoret:[1,8],theori:[1,4,5,6,9],theta:[6,7],theta_n:7,thi:[0,1,3,4,5,6,7,8,9],think:[1,3,5,6],third:6,thm:0,thoma:10,those:[1,5,6],though:1,thought:6,three:[1,3,4,6],threshold:6,through:[4,5,8,9],throughout:[1,4],thu:[5,8],tight_layout:7,time:[0,1,3,6,8,9,10],togeth:4,tonelli:5,total:5,tq1:3,track:5,trajectori:1,transit:4,translat:9,treat:5,treatment:[1,4,9],tri:[4,6],triangl:[0,9],triangleright:[],trickier:[],trivial:3,turn:[1,5,6],two:[5,6,8],typic:[1,5],u_0:1,u_h:1,u_i:0,u_t:1,ucsgec:[],unbound:[],unchang:[4,9],uncondit:6,uncount:5,under:[0,1,4,5,9],understand:[4,5],understood:[4,5],uniform:[1,5,7],uniformli:[],uniqu:[4,5],unit:[1,3,4,5],unit_simplex:4,univers:[4,10],unless:1,unlik:[3,6],unlimit:6,until:[5,7],updat:[3,4,5],upon:5,upper:1,use:[0,3,4,5,6,7,8,9],used:[1,3,4,7],useful:5,uses:[1,4],using:[1,3,4,5,6,7],usmg:[],usual:[1,4,5],v_0:3,v_i:7,v_t:3,val:7,valid:[5,8,9],valu:[1,3,5,7,9],valuabl:8,vari:3,variabl:[3,5,6,7],vdot:9,vector:[0,3,5,9],veri:[1,5,6,8],verif:9,verifi:[1,3,5,7,8,9],version:[0,1,3,5],vertex:4,vertic:7,via:[1,3,4,5,7,8,9],view:[1,7],view_init:4,visit:7,visitor:7,visual:[4,7],vline:7,volum:10,vtx:4,w_1:[3,7],w_i:[6,7],w_k:[3,5,7],w_n:7,wai:[1,3,5,6,7,8,9],wait:[3,4,5,6,7],wal12:[5,10],walsh:10,want:4,weaker:0,websit:7,well:[1,3,4,5,7,9],were:3,what:[1,4,5,6,7,9],wheel:6,when:[0,1,3,4,5,6,7,8,9],whenev:[3,4,5,6,9],where:[1,3,4,5,7,8,9],which:[1,3,4,5,6,7,9],whole:5,why:1,width:[3,5],wilei:10,wish:[1,5],without:[3,6],word:[3,5,6,7,9],work:[1,3,4,8,9],worth:[3,4,9],would:[1,5],write:[0,1,3,4,5,6,9],written:[3,4],wrong:6,x_0:[1,3,4,5],x_1:[4,5],x_2:5,x_j:5,x_k:5,x_n:[4,5],x_r:5,x_t:[0,1,3,4,5,8,9],x_val:4,xaxi:4,xlabel:[3,5,7],xlim:4,xtick:[4,7],xticklabel:7,y_0:[3,5],y_1:[3,5],y_i:5,y_k:[3,5],y_t:3,y_val:[4,5],yaxi:4,yet:9,yield:[0,3],ylabel:5,ylim:4,yoshida:9,you:[1,3,4,5,6,7,8,9],your:5,ytick:[4,7],z_0:[],z_1:0,z_2:0,z_i:0,z_k:[],z_m:0,z_val:4,zaxi:4,zdir:5,zero:[1,3,4,5,6,7,9],zip:7,zlim:4,ztick:4,zz_:[5,6,7,9]},titles:["Stationarity and Ergodicity","Semigroups and Generators","Continuous Time Markov Chains","Kolmogorov Backward Equation","The Kolmogorov Forward Equation","The Markov Property","Memoryless Distributions","Poisson Processes","A Probabilistic View","UC Markov Semigroups","Bibliography"],titleterms:{"case":[4,5,8,9],"function":1,ODEs:4,The:[1,3,4,5,6,7,8,9],algorithm:[3,8],applic:3,asymptot:0,back:[],backward:[3,4],beyond:9,bibliographi:10,bound:9,calculu:1,can:7,canon:5,chain:[2,3,4,5,8],character:[1,6,7],check:3,comput:3,condit:9,conserv:8,constant:5,construct:[5,8],continu:[1,2,4,5],count:7,countabl:5,curv:1,depend:3,differ:4,differenti:[1,3],discret:[4,5],distribut:[0,4,5,6],dynam:5,embed:[3,5],equat:[3,4],ergod:0,exampl:5,exercis:[1,3,4,5,6,7,8,9],exponenti:[1,3,6,7],extend:5,failur:[5,6],finit:[5,9],flow:5,forward:4,from:[3,4,6,8],gener:1,geometr:6,gillespi:8,hold:7,implic:5,impos:5,increment:7,independ:7,integr:3,intens:[3,8,9],inventori:[3,5],irreduc:0,joint:5,jump:[3,4,5,7,8],kolmogorov:[3,4],linear:[1,4],map:[],markov:[2,5,9],matric:9,matrix:[4,8],memoryless:6,model:[3,5],motiv:[1,3],necessari:9,notat:9,oper:1,overview:[0,1,3,4,5,6,7,8,9],pair:9,paus:7,poisson:[5,7],preliminari:1,preserv:4,probabilist:[5,8],process:[5,7],properti:[3,5,6,7],rate:5,refer:10,represent:5,restart:7,restrict:5,review:4,semigroup:[1,3,5,9],set:5,shift:4,simul:[5,8],solut:[1,3,4,5,6,7,8,9],space:[1,4,5],stabiltii:0,state:[3,5,9],stationar:0,stationari:[0,7],suffici:9,sum:6,summari:4,terminolog:9,time:[2,4,5,7],transit:[3,5],uniformli:1,uniqu:[0,3,7,9],valu:4,vector:4,view:8}}) \ No newline at end of file diff --git a/uc_mc_semigroups.html b/uc_mc_semigroups.html new file mode 100644 index 0000000..b858253 --- /dev/null +++ b/uc_mc_semigroups.html @@ -0,0 +1,735 @@ + + + + + + + + UC Markov Semigroups — Continuous Time Markov Chains + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + +
+ +
+ + + + + + + + +
+ +
+ + + + + + +
+
+ +
+ + +
+
+ Contents +
+ +
+
+
+
+ +
+ +
+

UC Markov Semigroups

+
+

Overview

+

In our previous lecture we covered some of the general theory of continuous +operator semigroups.

+

Next we translate these results into the setting of Markov semigroups.

+

The main aim is to give an exact one-to-one correspondence between UC +Markov semigroups and “conservative” intensity matrices on a countable set \(S\).

+

Conservativeness is defined below and relates to “nonexplosiveness” of the +associated Markov matrix.

+

We will also give a brief discussion of intensity matricies that fall +outside this class, along with the processes they generate.

+
+
+

Notation and Terminology

+

Let \(S\) be an arbitrary countable set.

+

Let \(\ell_1\) be the Banach space of summable functions on \(S\); that is, all \(g \, \colon S \to \RR\) with

+
+\[ + \| g \| := \sum_x |g(x)| < \infty +\]
+

Note that \(\dD\), the set of distributions on \(S\), is contained in \(\ell_1\).

+

Each Markov matrix \(P\) on \(S\) can and will be identifed with a linear operator +\(f \mapsto fP\) on \(\ell_1\) via

+
+(47)\[ + (fP)(y) = \sum_x f(x) P(x, y) + \qquad (f \in \ell_1, \; y \in S) +\]
+

To be consistent with earlier notation, we are writing the argument of \(P\) to +the left and applying \(P\) to it as if premultiplying \(P\) by a row vector.

+

In the exercises you are asked to verify that (47) defines +a bounded linear operator on \(\ell_1\) such that

+
+(48)\[ + \|P\| = 1 \text{ and } \phi P \in \dD \text{ whenever } \phi \in \dD +\]
+

Note that composition of \(P\) with itself is equivalent to powers of the matrix +under matrix multiplication.

+

For an intensity matrix \(Q\) on \(S\) we can try to introduce the associated +operator analogously, via

+
+(49)\[ + (fQ)(y) = \sum_x f(x) Q(x, y) + \qquad (f \in \ell_1, \; y \in S) +\]
+

However, the sum in (49) is not always well defined.1

+

We say that an intensity matrix \(Q\) is conservative if the sum in +(49) is well defined at all \(y\) and, in addition, the mapping \(f +\mapsto fQ\) in (49) is a bounded linear operator on \(\ell_1\).

+

Below we show how this property can be checked in applications.

+
+
+

A Unique Pairing

+

Let \(Q\) be a conservative intensity matrix on \(S\).

+

Since \(Q\) is in \(\linopell\), the operator exponential \(e^{tQ}\) is well defined +as an element of \(\linopell\) for all \(t \geq 0\).

+

Moreover, by Example 15, the family \((P_t)\) in \(\lL(\ell_1)\) defined by +\(P_t = e^{tQ}\) defines a UC Markov semigroup on \(S\).

+

(Here, a Markov semigroup \((P_t)\) is both a collection of Markov matrices and a +collection of operators, as in (47).)

+

The next theorem says that this is the only way UC Markov semigroups can arise.

+
+

Theorem 17

+
+

If \((P_t)\) is a UC Markov semigroup on \(\ell_1\), then there +exists a conservative intensity matrix \(Q\) such that \(P_t = e^{tQ}\) for all \(t \geq 0\).

+
+
+

Proof. Let \((P_t)\) be a UC Markov semigroup on \(\ell_1\).

+

Since \((P_t)\) is a UC semigroup on \(\ell_1\), it follows from +Theorem 16 that there exists a \(Q \in \lL(\ell_1)\) such that +\(P_t = e^{tQ}\) for all \(t \geq 0\).

+

We need only show that \(Q\) is a conservative intensity matrix.

+

Because \((P_t)\) is a Markov semigroup, \(P_t\) is a Markov matrix for all \(t\), +and, since \(P_t = e^{tQ}\) for all \(t\), it follows that \(Q\) is an intensity +matrix.

+

We proved this for the case \(|S| < \infty\) in Theorem 10 and +one can verify that the same arguments go through when \(|S| = \infty\).

+

As \(Q \in \lL(\ell_1)\), we know that \(Q\) is a bounded operator, so \(Q\) is a +conservative intensity matrix.

+
+

From Theorem 17 we can easily deduce that

+
    +

  • \(P_t\) is differentiable at every \(t \geq 0\),

    • +

  • \(Q\) is the generator of \((P_t)\) and

    • +

  • \(P_t' = Q P_t = P_t Q\) for all \(t \geq 0\).

    • +

  • \(P_0' = Q\)

    • +
+

In fact these results are just a special case of the claims in Theorem 16.

+

The second last of these results is the Kolmogorov forward and backward equations.

+

The last of these results shows that we can obtain the intensity matrix \(Q\) by differentiating \(P_t\) at \(t=0\).

+
+

Example 18

+
+

Let us consider again the Poisson process \((N_t)\) with rate \(\lambda > 0\) +in light of the discussion above.

+

The corresponding semigroup \((P_t)\) is UC and hence there exists a +conservative intensity matrix \(Q\) with \(P_t = e^{tQ}\) for all \(t \geq 0\).

+

This fact can be established by proving UC property and then appealing to +Theorem 17.

+

Another alternative, easier in this case, is to supply the intensity matrix +\(Q\) directly and then verify that \(P_t = e^{tQ}\) holds.

+

The semigroup for a Poisson process with rate \(\lambda\) was given in +(17) and is repeated here:

+
+(50)\[\begin{split} + P_t(j, k) + = + \begin{cases} + e^{-\lambda t} \frac{ (\lambda t)^{k-j} }{(k-j)!} + & \text{ if } j \leq k + \\ + 0 & \text{ otherwise} + \end{cases} +\end{split}\]
+

For the intensity matrix we take

+
+(51)\[\begin{split} + Q := + \begin{pmatrix} + -\lambda & \lambda & 0 & 0 & 0 & \cdots + \\ + 0 & -\lambda & \lambda & 0 & 0 & \cdots + \\ + 0 & 0 & -\lambda & \lambda & 0 & \cdots + \\ + 0 & 0 & 0 & -\lambda & \lambda & \cdots + \\ + \vdots & \vdots & \vdots & \vdots & \vdots + \end{pmatrix} +\end{split}\]
+

The form of \(Q\) is intuitive: probability flows out of state \(i\) and into +state \(i+1\) at the rate \(\lambda\).

+

It is immediate that \(Q\) is an intensity matrix, as claimed.

+

The exercises ask you to confirm that \(Q\) is in \(\lL(\ell_1)\).

+

To prove that \(P_t = e^{tQ}\) for any \(t \geq 0\), we first decompose \(Q\) as \(Q += \lambda (K - I)\), where \(K\) is defined by

+
+\[ + K(i, j) = \mathbb 1\{j = i + 1\} +\]
+

For given \(t \geq 0\), we then have

+
+\[ + e^{tQ} + = e^{\lambda t (K-I)} + = e^{\lambda t} e^{\lambda t K} + = e^{\lambda t} + \sum_{m \geq 0} \frac{(\lambda t K)^m}{m!} +\]
+

The exercises ask you to verify that, for the powers of \(K\), we have \(K^m(i, +j) = \mathbb 1\{j = i + m\}\).

+

Inserting this expression for \(K^m\) leads to

+
+\[ + e^{tQ}(i, j) + = e^{\lambda t} + \sum_{m \geq 0} \frac{(\lambda t )^m}{m!} \mathbb 1\{j = i + m\} + = e^{\lambda t} + \sum_{m \geq 0} \frac{(\lambda t )^m}{m!} \mathbb 1\{m = j-i\} +\]
+

This is identical to (50).

+

It now follows that \(t \mapsto P_t \in \lL(\ell_1)\) is differentiable at every +\(t \geq 0\) and \(Q\) is the generator of \((P_t)\), with \(P_0' = Q\).

+
+
+

A Necessary and Sufficient Condition

+

Our definition of a conservative intensity matrix works for the theory above +but can be hard to check in appliations and lacks probabilistic intuition.

+

Fortunately, we have the following simple charcterization.

+
+

Lemma 19

+
+

An intensity matrix \(Q\) on \(S\) is conservative if and only if \(\sup_x |Q(x, +x)|\) is finite.

+
+

The proof is a solved exercise.

+
+

Example 20

+
+

Recall the jump chain setting where, repeating (24), we defined \(Q\) +via

+
+(52)\[ + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +\]
+

The function \(\lambda \colon S \to \RR_+\) gives the jump rate at each state, +while \(K\) is the Markov matrix for the embedded discrete time jump chain.

+

Previously we discussed this in the case where \(S\) is finite but there is no +need to restrict attention to that case.

+

For general countable \(S\), the matrix \(Q\) defined in (52) is +still an intensity matrix.

+

If we continue to assume that \(K(x,x) = 0\) for all \(x\), then \(Q(x,x) = - +\lambda(x)\).

+

Hence, \(Q\) is conservative if and only if \(\sup_x \lambda(x)\) is finite.

+

In other words, \(Q\) is conservative if the set of jump rates is bounded.

+
+

This example shows that requiring \(Q\) to be conservative is a relatively mild +restriction.

+
+
+

The Finite State Case

+

It is immediate from Lemma 19 that every intensity matrix is +conservative when the state space \(S\) is finite.

+

Hence, in this setting, every intensity matrix \(Q\) on \(S\) defines a UC Markov +semigroup \((P_t)\) via \(P_t = e^{tQ}\).

+

Conversely, if \(S\) is finite, then any Markov semigroup \((P_t)\) is a UC Markov +semigroup.

+

To see this, recall that, as a Markov semigroup, \((P_t)\) satisfies +\(\lim_{t \to 0} P_t(x, y) = I(x,y)\) for all \(x,y\) in \(S\).

+

In any finite dimensional space, pointwise convergence implies norm +convergence, so \(P_t \to I\) in operator norm as \(t \to 0\) from above.

+

As we saw previously, this is enough to ensure that \(t \mapsto P_t\) is norm +continuous everywhere on \(\RR_+\).

+

Hence \((P_t)\) is a UC Markov semigroup, as claimed.

+

Combining these results with Theorem 17, we conclude that, when \(S\) is +finite, there is a one-to-one correspondence between Markov semigroups and +intensity matrices.

+
+
+
+

Beyond Bounded Intensity Matrices

+

If we do run into an application where an intensity matrix \(Q\) is not +conservative, what might we expect?

+

In this scenario, we can at least hope that \(Q\) is the generator of a \(C_0\) +semigroup.

+

Since \(Q\) is an intensity matrix, we can be sure that this semigroup will be a +Markov semigroup.

+

To know when \(Q\) will be the generator of a \(C_0\) +semigroup, we need to look to the Hille-Yoshida +Theorem and +sufficient conditions derived from it.

+

While we omit a detailed treatment, it is worth noting that the issue is +linked to explosions.

+

To see the connection, recall that some initial value problems do not lead to a +valid solution defined for all \(t \in \RR_+\).

+

An example is the scalar problem \(x'_t = 1 + x_t^2\), which has solution \(x_t = +\tan (t - c)\) for some constant \(c\).

+

But this solution equals \(+\infty\) for \(t \geq c + \pi/2\).

+

The problem is that the time path explodes to infinity in finite time.

+

The same issue can occur for Markov processes, if jump rates grow sufficiently +quickly.

+

For more discussion, see, for example, Section 2.7 of [Nor98].

+
+
+

Exercises

+
+

Exercise 1

+

Let \(P\) be a Markov matrix on \(S\) and identify it with the linear operator in +(47). Verify the claims in (48).

+
+
+

Exercise 2

+

Prove the claim in Lemma 19.

+
+
+

Exercise 3

+

Confirm that \(Q\) defined in (51) induces a bounded linear operator on +\(\ell_1\) via (49).

+
+
+

Exercise 4

+

Let \(K\) be defined on \(\ZZ_+ \times \ZZ_+\) by \(K(i, j) = \mathbb 1\{j = i + 1\}\).

+

Show that, with \(K^m\) representing the \(m\)-th matrix product of \(K\) with itself, +we have \(K^m(i, j) = \mathbb 1\{j = i + m\}\) for any \(i, j \in \ZZ_+\).

+
+
+
+

Solutions

+
+

Solution to Exercise 1

+

To determine the norm of \(P\), we use the definition in (39).

+

If \(f \in \ell_1\) and \(\| f \| \leq 1\), then

+
+\[ + \| f P \| + \leq \sum_y \sum_x |f(x)| P(x, y) + = \sum_x |f(x)| \sum_y P(x, y) + = \sum_x |f(x)| + = \| f \| +\]
+

Hence \(\| P \| \leq 1\).

+

To see that equality holds we can repeat this argument with \(f \geq 0\), +obtaining \(\| fP \| = \|f\|\).

+

Now pick any \(\phi \in \dD\).

+

Clearly \(\phi P \geq 0\), and

+
+\[ + \sum_y (\phi P)(y) + \sum_y \sum_x \phi (x) P(x, y) + \sum_x \phi (x) \sum_y P(x, y) + = 1 +\]
+

Hence \(\phi P \in \dD\) as claimed.

+
+
+

Solution to Exercise 2

+

Here is one solution.

+

Let \(Q\) be an intensity matrix on \(S\).

+

Suppose first that \(m := \sup_x |Q(x,x)|\) is finite.

+

Set \(\hat P := I + Q / m\).

+

It is not hard to check that \(\hat P\) is a Markov matrix and that \(Q = m( \hat +P - I)\).

+

Since \(\hat P\) is a Markov matrix, it induces a bounded linear operator on +\(\ell_1\) via (47).

+

As \(\lL(\ell_1)\) is a linear space, we see that \(Q\) is likewise in +\(\lL(\ell_1)\).

+

In particular, \(Q\) is a bounded operator, and hence conservative.

+

Next, suppose that \(Q\) is conservative and yet \(\sup_x |Q(x,x)|\) is infinite.

+

Choose \(x \in S\) such that \(|Q(x, x)| > \| Q\|\)

+

Let \(f \in \ell_1\) be defined by \(f(z) = \mathbb 1\{z = x\}\).

+

Since \(\|f\| = 1\), we have

+
+\[ + \| Q \| + \geq \| f Q \| + = \sum_y \left| \sum_z f(z) Q(z, y) \right| + = \sum_y | Q(x, y) | + \geq | Q(x, x) | +\]
+

Contradiction.

+
+
+

Solution to Exercise 3

+

Linearity is obvious so we focus on boundedness.

+

For any \(f \in \ell_1\) and this choice of \(Q\), we have

+
+\[ + \sum_y |(fQ)(y)| + \leq \sum_y \sum_x |f(x) Q(x, y)| + \leq \lambda \sum_y \sum_x |f(y) - f(y+1)| +\]
+

Applying the triangle inequality, we see that the right hand side is dominated +by \(2 \lambda \| f\|\).

+

Hence \(\| fQ \| \leq 2 \lambda \|f\|\), which implies that \(Q \in \lL(\ell_1)\) +as required.

+
+
+

Solution to Exercise 4

+

The statement \(K^m(i, j) = \mathbb 1\{j = i + m\}\) holds by definition when +\(m=1\).

+

Now suppose it holds at arbitrary \(m\).

+

We then have, by definition of composition (matrix multiplication),

+
+\[ + K^{m+1}(i, j) + = \sum_n K(i, n) K^m (n, j) + = \sum_n K(i, n) \mathbb 1\{j = i + m\} + = K(i, m-j) +\]
+

Applying the definition \(K(i, j) = \mathbb 1\{j = i + 1\}\) completes verification of the claim.

+
+
+
1
+

Previously, we introduced the notion of an intensity matrix when \(S\) +is finite and the definition is essentially unchanged in the current +setting. In particular, \(Q \colon S \times S \to \RR\) is called an +intensity matrix if \(Q\) has zero row sums and \(Q(x, y) \geq 0\) whenever +\(x \not= y\).

+
+
+
+
+
+ + + + +
+ +
+
+ + +
+ + Semigroups and Generators + A Probabilistic View + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file diff --git a/zreferences.html b/zreferences.html new file mode 100644 index 0000000..19642ef --- /dev/null +++ b/zreferences.html @@ -0,0 +1,302 @@ + + + + + + + + Bibliography — Continuous Time Markov Chains + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + + + + + + +
+ +
+
+
+
+ + + +
+ + + +
+ + + + + + +
+ +
+ + + + + + + + +
+ + +
+
+ Contents +
+ +
+
+
+
+ +
+ +
+

Bibliography

+
+

References

+

+
Bob05
+

Adam Bobrowski. Functional analysis for probability and stochastic processes: an introduction. Cambridge University Press, 2005.

+
+
How17
+

Douglas C Howard. Elements of Stochastic Processes: A Computational Approach. FE Press, 2017.

+
+
LG16
+

Jean-François Le Gall. Brownian motion, martingales, and stochastic calculus. Volume 274. Springer, 2016.

+
+
Lig10
+

Thomas Milton Liggett. Continuous time Markov processes: an introduction. Volume 113. American Mathematical Soc., 2010.

+
+
Nor98
+

James R Norris. Markov chains. Number 2. Cambridge University Press, 1998.

+
+
Par08
+

Etienne Pardoux. Markov processes and applications: algorithms, networks, genome and finance. Volume 796. John Wiley & Sons, 2008.

+
+
SK11
+

Prasanna K Sahoo and Palaniappan Kannappan. Introduction to functional equations. CRC Press, 2011.

+
+
Str13
+

Daniel W Stroock. An introduction to Markov processes. Volume 230. Springer Science & Business Media, 2013.

+
+
Wal12
+

John B Walsh. Knowing the odds: an introduction to probability. Volume 139. American Mathematical Soc., 2012.

+
+
+

+
+
+ + + + +
+ +
+
+ + +
+ + Stationarity and Ergodicity + +
+
+
+

+ + By John Stachurski
+ + © Copyright 2020.
+

+
+
+
+ + +
+
+ + + + + \ No newline at end of file From 41d22ddaa60dc6b3ab2db5051e331d551a2de06b Mon Sep 17 00:00:00 2001 From: John Stachurski Date: Tue, 8 Sep 2020 10:30:49 +1000 Subject: [PATCH 02/21] Update documentation --- _images/ergodicity_5_0.png | Bin 0 -> 23154 bytes _sources/ergodicity.ipynb | 636 +++++++++++++++++++++++++++++--- _sources/ergodicity.md | 589 ++++++++++++++++++++++++++--- _sources/intro.md | 51 ++- _sources/kolmogorov_bwd.ipynb | 4 +- _sources/kolmogorov_bwd.md | 4 +- _sources/kolmogorov_fwd.ipynb | 9 +- _sources/kolmogorov_fwd.md | 3 +- _sources/prob_view.ipynb | 270 -------------- _sources/prob_view.md | 231 ------------ _sources/uc_mc_semigroups.ipynb | 206 ++++++++++- _sources/uc_mc_semigroups.md | 204 +++++++++- _static/jupyter-sphinx.css | 116 ------ _static/mystnb.js | 44 --- ergodicity.html | 611 ++++++++++++++++++++++++++---- generators.html | 7 +- genindex.html | 5 - intro.html | 44 ++- kolmogorov_bwd.html | 15 +- kolmogorov_fwd.html | 9 +- markov_prop.html | 7 +- memoryless.html | 5 - objects.inv | Bin 897 -> 938 bytes poisson.html | 5 - prob_view.html | 512 ------------------------- proof-proof.html | 92 ++++- reports/ergodicity.log | 106 ++++++ search.html | 5 - searchindex.js | 2 +- uc_mc_semigroups.html | 210 ++++++++++- zreferences.html | 17 +- 31 files changed, 2560 insertions(+), 1459 deletions(-) create mode 100644 _images/ergodicity_5_0.png delete mode 100644 _sources/prob_view.ipynb delete mode 100644 _sources/prob_view.md delete mode 100644 _static/jupyter-sphinx.css delete mode 100644 _static/mystnb.js delete mode 100644 prob_view.html create mode 100644 reports/ergodicity.log diff --git a/_images/ergodicity_5_0.png b/_images/ergodicity_5_0.png new file mode 100644 index 0000000000000000000000000000000000000000..29654b3d781522bcaa36191e78a4557769769ca5 GIT binary patch literal 23154 zcmZs@1ymeSw5ty3z#rj({s^?R)7ubAP|4yoc={(bRn1zj7Jm1ZO!>V#Q9`C`XI-vS~zxQ_WA4GqEVLfv{bMN$v`w-%fdnl zD084vvjO+^n)*jBzWH=)0X5EUd1;Q}-#C)pg9Aurk1ONAU&PuiwtI>40tSbZ`8Q`P zsRBMvr}qvSo;{2@?aWU5vMCJO-^kIHPOJpqA1B)0o=%oJJ=n7O+-%oB{iMc`Z2DMU z3Zu^Ol36aY5D0|K)crtuzwX5Z*x}}4BRBW`<$(Wslet-Dq@7|Yxl9H2O zcda};JhI%kBk1+IJ1qpW3XHii(td1{<@&wxba~ywY_8b#h%dOraBqh%*W912-23l@ zgSzxY@m*#Wy`PRs$9(a{hs6GKJhZg5)QpTUdF|~P)6>&k-MU&@(yFR+zC?xP_g3NC z+lIggGG4iVBt(4JDz?#PrI~Z_$B!Rnm6fX}C&|glO#$spANN*KRh?-e(wYs2S$O~Th#01=aZ6R-<~pkVIvdB-RJp4T(G zET?fE=c8$gHNoZrV@m-Fa=hwTt^dI}X6EM$B@3z7eCE z|J&iZ^)1cuq?H$VBp<|gk`1~jq{jKVKDMCxALk!lpPiEieEmyt`Tt)u@CZpq>~QO* z=2LfeB4#t~LK0QByoF1!6WK|2Rf8qeS!h(U|+3X;yZ|_~qd; z%V7W~%j>)i-{)rPu)TZZK;|`V`B-JEd>4C;qJlczR&OV6$lhAo{tqo3yn5(T>MnCwB6Dj%U)pSIoQBwNSh;Rv=_wkImx`f{(}de4vF34xSzIZui% zOSGD@XTUnD7;xh1G@_1=EnP3xK`t&F@sd!34tJJy#eDAgBirTg>%Z|T=pOb=&baR- zGhP3{G@SdkNX4ciT^naN+n~n|M*J~ss@j3<1Xur}rwrsbi+ygCOZO~$Zdnr(3RP8A zG1AZ0E7izfw2V%V{rpaq)QoWvPC>d#cLzbTlcEt5AbQvXEX_#Ca-nfD<3V#yieWyrW${4jN@XE7&zZAz9 zpHMetax^vVx=^3Xn3!aJCQD)Ub|(r&J~{JMLWk?iYds->lQ5j0)eo_vM!==luqOes z0GCrJuFU9#Ph{}rgRL4LRaO0Q!{P|;eSlZIkGJJ4H zP_s2Rg6iIVb^m9Dn*GgGHn!lKSyS#O3{29ZBF{b;OHMl&@)#5F$wCP}N_m0kI-^wd z*p+AZ-5bRQYm6SifV)M;h^)BLZ{)-V6LVi*YK8Qqhd^9i(AcXEgXvy#bj}3eUx>V! z#k2Ve;R7_$t6!h~#t%+c>wIo7=hKuqNc7Ir-hJ<-Uzi7$Z!!DQ3J+R<`+`Zr&~SGm0%tlQ5F1aHYc!cB zio={|WtG~ap30!GVv^}8#oE)&DebBlP;Oj2ds)A$QA9LAf_D-@)x8^n<+~Z{hcE2w zyGf;o1*W7lc|^>`uAR z@Kdu{x1A%1;b3ccoFAT&|E`z7;obCuxkjeeXEb?joZ zjhI_fkVT+{a5m!v+xX#t$HDFP0G_(3m|NL#y(()wg6y9b6n2#KRmH(iJ8{vsZ_nE4 z{Er|)-v^O)SBABFu4CR96Ey3E7zGP>bKDq{Ee*l!U8fNSvwve;9UjN!Oxm?q0~IS| z`Wci{=1Cg#p*AbD{K;yCKtOCry3SAxw&FU1?^!pRj@f>T)g51R`&1;K8+7;Fl~H1} zdw{;uj4UxT8AwuOLqdSJ*n!CPg*8pkj~Sh^Q|SFohx_GUGsA;2yY3pJHjT+yX5lQ? zx3+&=dG4Z3%*j#7?FJu^4FI5unwsfuD0r30^Br7Y7}M6%iTu_mTL;0vXYpRJ6=5c7 zAw;O&W_$cnR|_efF3)6#KIB(yK3ATSr7Iaw#?N!KGph%x#P{|fP(iAbQn>B8Ye7J_?HLzP6ZpL~+_S1gPvEGqKn zaaxyA-eah+y%4F-)CEHOs02d!QstQ2n3Y(W3rW(o$z%selo~00i*+?qU0yf~dVY>U z7gX%X22iLOy%B?T6+h#evkw#227Sh-<2|IH+7yPMs*r)2(-d8lw+BUDj83+OcC7-pl#?E>Yy+ZxSx zpIb+@>dqpmwcqLVE@!5e?mxS%liO#tsrURpe0W-9&C`Z+lJ57gGY`$(1^$#iEZw1YA`Vs%kX^lQ-+Q_rLs=*@LL&XiBipO^4Aw< zAYK%72~8gUQAFXnkwU`UNKOPvM-am0?d=7e5yx|Hg+?1tT4Zdldrg?_{ypHn=&ZiE z@U!={98myZwmjDJ4~G9TotGGC{PJzwmi;0ZmclqpeE6Tg@imU~nqOOAu8d~xT0M0I zxKZxT*A};!f7cV&6@NDZj`VM(CM1raFEj?mvtCL9X!hGb=>&vPeANxlzZn_0hIMIL zpN_y+T!D=3!s|-ERg(EseE%!Ni^PI~=@V!nt{J%*S#GD@1bm#Q6xJ+jDm&9oH#t#Q zv1Np_?Xx#Op}I~pj}WwJjQL;gB1>^kQRyL@me=fTK8Ew|=kM$cT|Hi7%#*=hWr#Ww z6C-^Qf8vp|X|)HLd>tr>{5rD_a^ESlO`fFJ;BAMD=-OBUXM(xNr1zDL(llv3IAA$Y z(jN(ZZj{%3y%0Y=fTq+%N$I{_csFKSv5ElkqIhl7!cPt1OAA+i}aSaN(- zY+(e=xg(K#{;y;8s$J|$;!VsAl52?63u~#WO~h;yeyL}uhlk05ZvMf%qgzKs?Q?b}*Lq^|J4Y5DO38%V?kH;3I>e!ewo9&#i?Gsx^Z5)16? zgTfh<$m?wcU}Y(1@?aOOnG!3UbZ~LqvcW%@s_;vvrT~ zedPtRveXkHe;B!T&X9>giG^WC$+6}zDlN0NhNTdto#j{K^K_nT9Zu`;eO_9o$ zEowx$#ae?n))i{2uTPL9FBl+knzi%2=8;LY-OLYTB>!@?|Me#vRD2!vB8mp*?`DV5 z{jr3qi8=M$@Fd<({QnG=iTKpZ8COR0A=r?W21ydZH8(=Rf$iRR-b{bue>yCs*X56z zp5~u8M*Rf_5rBG5PHc^r(Npus?(&Jd%9eih0PRc6L5KD z)#8+lMktPvc|2NXJYtuft6fjYCWp3qK|Q~&$ck#MUq1K91#TC2x~V~!NF>`1!Op0X z;o|WOtt+E>!eQ49xdfxL<#tVrPSlJpk9G5f5KcrNNCJRnS9qlXHZ5^-?qX+KD8NdY zO@@zdP8Q})9KUD4ga%MT;pSQK_G{f`q_k}Ka$a;cq`$viTxi{RBz--WBbRku8Zcy; zGs%RpOtBWJ$FKl^>Dv&1C+lBK{AlTRqOj|dw)&o#tu^dwM#?FWAVGo7_`yl2aLs?Q zO@9Uk5?>$a^@6xwqm|{)`t!p)R~WKoj>qd=@DHe*XlLyeclN9i*j@3J6j8_tTiULM z_im16A})9Q_FZ_qJ_mr2vUmdsM@lF&GVc!SILeaUdi)bTjoAR&m909afIa)T9+aB} zFUx^a-bhYEkx11OQ34GFAsgHIg#__bAb2nt384g3P&j7Bjq<|7B^6q1U(hGtS=x!x z)6C;~o1Dt?1kf6D-Z_vP`~ZmnnZ)#Z4P^3*(ZO*T|gZDx!&hpE_zry(M76D>9cHf@mlZcjobUtXmxE9_OLiIY!;r{P4zT0;*-^Y;_ z?OTuJ&FnEhXpV%^jRz2D5J)53{p@u^|8 zvg|b4&J3nL=dC;VYE@71sk&M-KPM{K<<_D<7bW%YI225o%h^T6%|#@5=VMT;bw5`J z(p75=oGw8g&6E}{?BQz|%JYMtgW&A(-Ov>1Xz7ty?bh{eyFD;~tk`nwL zy%tYx^8U~*qc%`2Ps7d4QKsrX(FVa1{rZDNbmqc?TzUe>Q1|F59Ke@CL(%A%YCX-d zkR`@_`tL6pk=NCW^!2V~s*M_K2~cxB7SOEM(uY;l_(u1k&>2k^C3Z2EIs}KJo`W+% znTwF}o9j>>wGVW9VEy~oM|62PxWZo{YSI1uxPI)o7iw-;Wg0--BpN&Ph~Aqcv*(Ko z={W9Xg79GU5veS-T$;?EGO1{`J+IaSF*@Hs(MOsD(oeH3VicJG0`W5YxG5>K&VP`@ zDT%#RzjtUTi1vj&_MDP@3kZ-uCPMprivOTdxM=9j{$H|B+s!MpN{QBq(K^jQ)xQV! zc2ip8akpfr*=kP>4V~V`@+%-hYyk&O)J>4r^;;IW=_q#Z4-CAAm5uKE_AkZI_+=yW zm*)6gG_0HUib{K{a=NC{16$< zbOTSMsxh3QY?>v{JhM|L=}2z2w2rAnk)8XklCiTB+rs9sElueCPo&+&YR83Vj0{Qy zotT37SDnMB*im;kzn4oLY@REP>b89j;1&`^0PBACHRA%wC} z!X7Xwv>SpN8k1t6W3Zs>g@A!U8Hwq$?Ls3>C+)iQwB=m|h-$~wOtH+~4`>6if)U`5 zp6)twp5!%h0b&}MO%kEmJXJh>lWAOb%s^I`7kG8OwKeKkHArqW~L;2ERAIZE^JN-oua|m zMgM7wAToi+dd1*(Fc&~Lf`<~H0e5|(aASnS*2g}lUqa#+1M2&+tuVLEaXxh~9Y7(b zsDMOQF{ec)D>hzkoI72$aJXMPD6r6ca3g~vCiw!U6GwyYdyFQ+Y&6i;c{R+d>tRy$ zvEcqs>Z!vo2$@nqErA-#e?A@MwlC_uEsrRj%Mi1GF^oENj)PojXjd z%myeSW?_R-N3%Q_{a#PBbbVa10ekS!oMjHS{Z{r_t_IgQg*d2R*NT&N$i)(L6kt613f7woVTCoaN0&WPJD%Ia54&n zj7I@@psD;qWlxs(70JuXfrP`FOHKI>Z)BNfJx>28rr_y%-`A4~_uDySpq|nfA?X7a znVm!|Myg-5C-*c(Om;uO=Z|N5IJiIdnb7Zrtwu)CQFb1-1VLtx$?q{og6K8sBfyms z4qSM0P=}%yLi)P~Qti3)?hR!4+Gcmxy*3U;m`1u7C8QvGvPuE)1zxtkV-ZVCZTVE^xzP3uXupS#YFX<1($c{3k<+BS>p zecW$R8o!fhN@r^H)N4jAb7UDwq~PU1PrS?_sG}J!gno}BxN4OTEr1z7yQz^9q9{-$ zdb)5FJ^UiRgz-h>+fka;|rUW_<0t-6#yelmdos1Ggj+#ViQkDpw zw&8;|AL0qiptI@ALoN`)`*CRYcg}qXt5Sl$-%I@gYBC&0BEjmgeR9mLu9fs2n*#Z7 zlsV16_ zh0_qFF$!3V0?}k*s-Ea)NctL(NeJ#Y-ap5F<&0{!x$a`5Vv~YxaiI}*S}mr?_%}8u zjeDFR6O(T=h~|4s*EHKN|Kb=kUR`KH^m1iL9St0wcG`cWD|fgse5GR-p7fDAa4Hj; z2wA9-a~Bf}ir~%jj)|R~w-aq$Kc6nZI+~Pb5Xvq?rnwj075X=ad!spD;{<$oJ$2+v z6FK!e)@rBa8rj9wT!tu{Y%(W4bh*3b1!^XBAo<)3br%*B`lbgIv#&PW7yuqI9L<$8 zoNnfN-JUgiK;B_2g?}57VwysTSDL#XjmUU721HpI*NDJos&4NozgAMEnvRVlP;%k$ zKfp0|KbQla1Uo2)Ge}-1uIt*&oM_wq$lm?lognzKg$M*nz1STg0}VS!`GU0}*n}sF zpd*m*D=)Lf^pI}q4pB@9pKbc8cXq#=;1Wk@IxRN)hm6+qG$;6Qx1yt)S~%Nhcd(ZQ zC;Lx6QE!QhhN5NlinA_F|LLY-)@m8-JP;ZXh>Yz<%EY-b&*?SMxZnd;HEMVrck3E0 zlmQv@!c*tRwi5vVdhSn0{{#Ko917&A3^H3AjL{ink>Rj)Q|D-b1eWtP{6i*Et>E<3 ziP<*#P~utu*xw)Z$Q>kn{xm#{(8)NMd#YjhY3Zv`n$koz7omv>^k}*hAv$HH{=s+| zU{RVEXW#LNUX_=V!r12j8|vRo{wPyEZ-$n}^98_9_V~F0_2(aCuCijvbTMet(90Rb zg3~%z8;VrDH(EBc9MO&LY~cGgPT`vPUd>zcaYnCO%jUUEhH<=KU7+O!Uy~P#sUJ?Z zr{Y8d%GVX2WIQ~8z)od0_$`HmJqcvKmkHTZce37W8bnsqt9nxTV3BjB#G@JY22)T7 zuu_3U^Ztfjl;<^R=?$kg6pmO+bFG`S6t6lXY80t60t*!oE z=N=m;N{%_X<6{7RZPs8>s7ukLbJ}q5xoM4Na0qpoTI@D6@8w{YpzP|WS|@bO?(*k& z9?PuzK11mB<3qSJSfjtd*fcA~Qwx+ff?bhA;uC`J4D$S!7?GIBg{|Bz^1BvP4EEbz1B@Y% zp)lVjPloy3<~73{fESC$mGo`sGMO(g*MY45PZ@8M>SQ`DR-=Iq`tRIHF$9P}UU*)l z6d6#S*WcFV)|wcT5_AD&ds625btjtTzAm3AB_XA<(G-{tnU+ilGMW7jUeK)F^!B{7 z22f>bD5kAS78H2y&j^qQtt&7O55OvX=k8$_JsSWUmxdVu6hhXK{|DiRbmfqO)2Auu z*65OV9{`HK-oNwn?w zXE%^WZ-WWM4LcHj60qV?_B*iR)GPu0FUM>AV2G+mAArD*kG!}GjHi=*v(b&i!T-Pl z=^v4Z010shOo(hs5#EnovhPnB2hP#S0$x^A*r(+qql7viKWCa`a1LDu=WcyjBOea)+@CB!sxG`aYu3K$d?s z9=*mt`5KuaUf&LPc3?t#^nVhv$`z!QF6iR*UQ4;VQ!#!+yZb67+F{c%+`@*2~{WNX1!gn+YTOz zFdBNiSQp-Lj@vt@&Qw~@dJd4|Fh@#eRm6td{vFOj38sTZqYh@~}vw8y-1x@7i&-heK^Z>nX}>xWO2>VIsEkMN+UZT4Ud#{M@Kusx~qJif66aKMf!hG;D3Rbazgd1$rmApeBt z_8>XE@=OxhtvrEqQx9>!dKVxA#JX@?1dx(@Md}#pm$?; z{|^_CPEZaiSs;f( zHg;vZ*%p}%sP-u3&n4d9)&M3~E}Ga8r^R3Fa^(LR0(5qjh!O-tFb+?JKg#5LnF2&n z75M?0SQGP1Vf*s#v`i|i@!$CG&QppXxIMVOJ03NM{`~hC;)WA2@t>j*0=KL&Z9ALo z5UlR&_Ah#uCm!6T5Xv*`8!q6+q%(~;e6G-6J)QNp`xAlhUV{@6O3>AHJvy~46CfRF0aOSYO*dWd{Peh+m<{xV!2|pOmmTi@3wPKN z*DiJpQ7%L8H6=mrHuy^dAPk(T!vQqR=6>tcp08vZaWage~<{hp#}+8(k*7z@7j~1 z_6Gq*9|K)Nq(@v+rK3Hxm)bKtnhtrmq7K5oMATjhXJDr%sb#ijCy>BDcG2Yd`0lM} zF3ziI8LLi4aVa*8v|)kdR0(VWZ}^4pPH2!Y_<&X@4at+HQGl;~D49Ak4I!wv``jIT|?f#M$WtNW?G@W5=oLN1r8C#}!ZnsH2IO0vHSz%{AcY zN#cShxff5S9lrr228ZJ(6F`u99cCX(1bnS~cUAPC9P%r%09oMWf$MO#9GPovb^>tn zZ>&wx8r+NKi`a|i-1Vpd?b1?K?fV&t<<-p|yYaDIj1**#m)`-DfPeFj*Gf!E(0g;_ z=y28ipb)R!-qeiqZ45Hb^z8J`aVF-$os0~n*r9+H736W4|LO5z#|E%Zk+2*L0B~z9 zA2edFqU9?0okzn)c2g9M`A>@2$#fSFHe9=xj+Pwn!{+}{qPVSr9Q*BZC-X83Nw(Pc ze^QAu>Ob{Ls7makj3(cuh@DG0*t^^DWDvz)^YEf z>8N|^2Vm40@Oe1kDYsvNs%QrG}*h{m_q6p^DGNQ29twRIkS(?*E zNe0xL8=VTQ+9Vbk?`P}Hw0hf4Oau32KO=A!>_JeL{jnNQXz0GHnT#Ez3V!#xd|Sbc z-(8pLzB`Rs0kx_;&{r|&aQy?g3Jnl*{6R3cSMB)4>9VxpmK++`X@`s0l7V0T66&Uw zkbkr&RaDaGbzdSDiroh$zIyXurDc>nK0(W*n$K=ujp*$qDWmNDtat+0bpp)X@;Adc zb4^OB*I+oFFn67d1bu8k97frHdww4(&9%L62h>X!koPz=Wtm#Zj%0O*Ek$v0h-|he zAo@{T)XP*>*l4yBaRe)ifHfKTQXVZgB>}A%!%=LYFtY;^snb__@DllVSiv`J6CjbA zXFT~BL*hlQExXD>^iQGhq`^GaXX18^3iX8&NY&^Pv|f+T0cn9f5NJ1EFB;|68UFc3 z=xNTJAGZaVI76jAq}Cpd@40nlO9~=s(ei~A9i9Xz&Ye8=l3nOE>*qhD2EBuH?^3~-36``u6p-bc+ZbK}q0fO7# zXmRoHuu$}9we3H;h@<@?Z8I8G+~U??_P<0bKK-&+BevxTD#Fo!TZ8szO>GUc{S%3d zs`~r>HQmcYKzkohxE^oLtEx5PZD^@WwHlOvv-*SlSxZY7ifNF6x{bzq1#^e1=W+Bz zNh=yyIW2R;>i?h?%JQlsi1kxFw=VsYEG*3Mtqn%-iu=yuAG2s0DIpzA0e%~6W)(t0 zW^XUQMlUEi8To1CNH-Sp~-%TVvyO{9O!N+YWWBCeXHthcCjch zLL=T_{8Jmq9i#l*V!PtH{Z61_Z<-?d$)<0%+>F(9)RFY_2ymuZdU%Y$nfm(V*ZD<*G=Th`_UPyE)uQxxyY*nCxlNP|L8 z^mS$zah_{0Jb@7@BHV^RZOToYF1K;{xp|X=8ko)od$rKxJ5g_P@`11t<1nV2ZbReg z*8&^9+JF8xQF%l&E8cZ4vRTnc5idqJDEusMq0_!rUo5A6MZ?`&*!Js)5YZ)w%_}xq zR{cLxm`-nhgk7zT_J_%N9j~l2#u%#qFaGl44|xAr5^h4X4Oy`mTz~<#H^%BWWbp0a z@XwO65AgP_GhLr26#VRes+GFQ2|S{{?FhzfFI(x2EOXuYeuX_S*0UN z42abLpvz`uAK-}7V&!bwiHWZ4pg!iPeg^;9P35jYYH-7dIvMmJVv_dheE5lWD(*F0 z6}PHX+s-n%zHFr6AU34+5I?rhHzx+c*;VGK^$m$77hAVQz zvTAo3jlfA}lq|#u`u<^b)kmd@)i8XSrWE2T{R%U7q`Am8Ufyoig${Q(n#q+0)QquA zJ@i1orcs$emgw>%vWaQKNB!5~)JQs;8L`ikGeGi+16Jc2tste~EVUB?gz|Ev_`Etx zE=Wbfv41VkQ()h?w3pQVehN}%kg*_G>u50vnt-L8T6@`^IG`V5zQ6=S<{@-Ngw+A;L+hp9A~h4Grw$l*MpD7!)4Y z*P1LfrB85J3O?zyHyZ4Z0z#d%)4ua+DjS!|~NgJelf#Yyp%@5D!6LK}3a3Kz(Qm!Xp~1}$PI{Uw%An1&D> z>n?K`pPw!P3H+DFhx;FmkE(RfTK_3MDIwF&H`Dz)v(J-e_q#{Euo2Z6``k=Q5pRM7 zdxrcflEE^#wexu03MoHN)*P@n`{Qz*wtv6L(%Fx0D-Q{-D*%k?$>h2a1=|XjFfP7+77Guz}-~>3H0q(P*j8*m!@_ z(@mYG_w^0RO$<0Xz&b52?cQ!X=r-E=o&V(vQ0;3ew^N?6hgkF}^Fb;Q_}8K+LQ)a- zH#;*rUrqf@;*x{aN-1Dhz4}QBjOZ`|UEi4#N5=$U07A3b`s+tqG7{@kVdbfib}r4) zBwC_t#&*j}o@uJTW!^U-6`}8?qf^|GotQr7B^6BEHcj-!IG#7=gJWE|<3LLz{Es!D zlQ-_lsrnW}%uoru*yANqR_*?RWh&Bc;RGt7IW5O&Ic0A>WINm0e6%v~0wJVYgD3-c zc3_v~zi5CnGi|?_Hu`9SXg0jG&GyG0Y!t}~PJh`9BXVT}WOdVVe4+m+K2&r~^T4U4 zy{E%qv{sE+uJ%TWa)BIw)C|-n{whx?W)DLR?6?q!1qR9i8SyJ$O2&gYKx%T=w@w#7 z8}x@41K15HK=x@1d&bH0IgK-4Vtv6nxDZ=hG^1DQqrg#Cr{^a~3V|C*tTaEaI!zE) zcinLekGy^eng<*0j=aolKAx`J){aXiHICyvUjcQ<2oR#~k24#|#7<7 zd@aDpjvPRtH>9&06L|^KPinz0Tfw zfgDB{N&(f1!tZ4#6(|hRk!leCNmBwn-UN#oHDyJE2J-MHWHfn>GIb#i;U0` z3mmMd+dBHRdIhHreGX!%#a24bADeeZ_|AgJI-EbAJ=SOM`pf!E<@mj6=eTdnU^3!=Q&gNhab(RP0z7xX4aQi_X? zH+uLEa#kYo?7pWfB18OrG4V#;dS{E7I9tUr`3MA;xl1J6S%5|GCn62)x;ppjk{lfX zzNXpvV6=USJp>vdK~bR)K&WnC&i9jx(dJCiN=ab*07!F#tQdiQ_`a4@*>ZK)P zHFPJ%7fk`xXn`(W$t0UIN|DFr`y%|^QL02Yq@T&rE$jMU-wK<>bUC1oBh8Gy6jS=f z@N1?1#?G+6BLvE?4hk@~Xg>Tmb@BT0WE1$ScoUNWe1 z@#UnX?wUvw_|A&n9v>`9v*(*VoG3QD8zO~-fFUWq&q(Lbzc=G0vtk9F;+eV^<>)bq zzt9$ym4$b9dS6{#QPU$Ss8}m#K>P_&(0~f@Kvh`4mEU^Pp`7qp>WrL>A>fF@f5e~o z%9X}8!M`}8m9X#|Da5-D-m%LMi3cyO*+v<7Fy%Pl@i(GX8q1E(+rO~S&gb7(e@?6= z__h;d3(bhh1si)jqUicQn#@;fiq`0MwLWY|($LBHA=VX;j%M+U0Hg8mAa={G7%xmn zf&i3WVIl2%CsMd#A->gE8(}-hZ}7p(Lb`ZT!th^`C8Z z*YbXzvA`4z)!ZCrGxGUv9@r!y7c(FE+Zqftgs9VX1NmjS(yBLu)T2gHYcg1f*PGs# zK(#~E(MUxN>|zkLw6=kgX=WoA0dMBy6#Vq~tU3B8MbiUC^ANc0g%WE^`yb2N%}@s~ zk8}}#NP(eO=p2_w-WX-R{0Z3OSBmnn1??eqy))y}MGQdv=(g)~)FU}+CS~Iz9~tSZ z)KnHGCO(|2uvu>Zt>^o=w_FQ_MpW4Oay0^=Pm_g5Oc8_jPkZ%Ma=!`Ob1yAE2Pd?a zr2Dp^cNhEXIjP6H*$xdFF7zVkW}0vqWLY{Pc5_9X+Qo!}IQgi~UC;n^63bPwRCgnd zBfT_8Ses6$JY8P1GpWFx1zPPhv>J`DrhiJjGF1Zj(c5_f`gFoq3g1uTz--;_54xO# z_qUB2K(z+w)?kZ@esdrw(-ND-kf!HB+)wW7oF3ieD{4YV#a7r@?VW^14`y>^kWHqw zC2b$~ILd0(zAG4|h>)hH06G=n2t1Xu?VVGNWHq8IpI~e(DQW8=R z7&+3pFPXAjwXi(z=x^J}^@f*vWu;NXT9Vc}j6V9gGK@T-B%(3qbq7+VxpK>&b3n^= zHj9>hPuP_;0U*^{i_$#rQsVCazNN7Pbv={Q^}a;>`E$FKMDzE$PnEck zK%OvoTxTm%RT zs-FW-Cxbg90;h~PqbG;f;Ip%}hz69kuq47&r6EKe3X~x%I%`KkdiLa~92`Kg!;%-} z?J4~{-w)S1Ab}|t)|~-{gJ`aA?v4~ivKsH}_&{#(8lcvaoI(eW`4w=LdDgu?1_Bvbxyr7Oa1}C_+o9bC+2rg8oHi?ga7vSzS}$SfZ#!X7@_QF!fTFhmuK@L{Zi5NG=#Ja;`EKV z7t?!$cCB_$c1GWLt0=UB4UP&dJ}pvYxB=>v%{Dl=2q}962|f~jpd^w(AtEd?tPXF) zKWFA5yrOd%O7wTfwy;nrhN(R9)_i*FqQ-bF4(vVD7s9GT1_&j6Zr)G=p~#kB5zZ8Z ze7JPJ>?JFBJm zrb|~UH81Kc=S+cN5z?zG)*T-ynP>^##YGi#BdpJ^iVU=6m>4P%)COCq!c?Jh!C(#$ z{pjF0WRgj!lq3Z}Nlr?69j+;C7|(xdj{3WOMNGPxJh5t+*-I;Z2-D1Ddn2VUr{s`P zpUO7Z2H(HFp1(x)5l4bxE?-Kprd|S><+o>&C7B4QHndhzid#V(REw`&@f$^jtdSY~ zvnZAW(q}S%DWMFg@IoW>-~FhR!qyP1p)pc$hy-C^n8JX@VgQ%v-w3CV9>)n0nYX9E z$QerUnFVrG1it}w=2UuC;76JddkH^&qAjatG&a?0#l@`7q&aC}wa(f(RkX_HA;(uXW-5Y(vZM+nhY?b!PR_ z$BWuvZnBKIUm4`UXU$P31}9w*5RjG|X$mSU$+CHuZjO}2HoDJ-My@>_rCh1W#qtRX zf^?2uBNxcN1((%jYUDw%wI-%@W3aVC<1B&?FtT^y>Tn-s?73d>qCuIXz-_fMXwsmv zvgFXBcR@D|R*OlaW<->z796Q!kDFdSQgQRfB!?NL>JQ8HpOS_DKo5uYG&owbzu6Bp ze_d-z@1{2a8jqE_T`}6W9*L#70zIlZY^8|Bhn{)Iq#3TO|scfDDnb4 zS)wSHrj%N*iLSV<5&k-m83o{l?5`I z0@|qf6N!wV(BDz!-uR>^ZEUoJw@t&{*`cPu#L$8q6z@&@i3J&H9P+yM?G?blxU@+}1+KLnjK+_d?4h{V0_ zMEcN9R($vnhC2U*j$8KV;Z&FQ%On0E2@@03PH5?!u+4V`b#8o0!q`%q>%jZLE#3HP4af$?5>ns9%0x))Wy)4@{`5iyM}XU00U(`FlLK zalLFs#vGM`qbHkF1N3n;*l{UvqdEDQTY#3M{pN^lnmrnp&a7jHd6C?*R5cNXNZt}A zX+5H17eSRHx{8gMSTpgc4l!9?x;}RvpT6WI1(dNNKJ(+0ROJLP6r98jZV!RBGKt0@ zOThOz#-T;IvU%=MZo*zvG2v4^wrC1k0s-6*=a=bF2r(Pw zLcI@UAnvq~I`GGS_}h@uM>IlZ>}_)M{&qmqY02Qz97WT|lU#AoE^dx#r1(7_r_|?) zP}kJlAL)r|Gn1;(VZ;*osSlpRn4X=jWy1-7=)>H-0Xk_s4JOrUIs9gO2L~;GcUr(O z(p-S&yMbp(nWIYNN6UZyaM}ua?>j0b8Yq&YDy`#P&`oru$FroJq-!ZWTMul0mJr)B z6>Eer{u18CFksA#+iGAQ_mE}ZHy!6iIgWk$xGuyhxi*;u4vPb(y-aR!X=sC=HxDI0|HdrtkL*CVSx7ZsVo9^{+WwMs zMkT~{VM(Liq6X+tPM!aLD?Hs=y`gBhQ25Pv@^56}sp!vR#16uT^?_^+GD^mPIbdoA z2E+YPr|J1QVjH|tOD@9&6tPFlMgqJkR~zvP8VQZhT=*_jEYWdd4?Gc?MG^Y!yE*ek zBxro?+~=XIp^0cELlYF;OvJs+y>P_Q-bGU=7S6xt`N(b=C3#8A!(!~#X=sPni-K$~ zh*96UejRe{N^}z|TqA$$#2@=x^V= zOhl?P#quSTVRj6~d^2o)-Wt$)b{gQ(^YcLl%2LjY9zSRbilc0L05K4Or-MNtP$Gn~ z^bj&v=TIo}XF^|h$;9V36LN?S*WY&xwMP;+Hi89b>nL@W>{Kw*vcUyAg!b);i^KHA z(B;L>btTQXi6hJjRnnp|&VvZd&XdS+yCm1kq@u7S*X_jDyg^gHx{ky_Pw8L&xZxwN zt$riEg7lcvL;LxH6_DYAeTmTM4z!{kk&f)&80j!*iU<_GRqW$^e2Y;3j z&1HdTxGPW2;jclX6oOD?g%(7&i}U5LZ1>t+{1~xP+)tpbTQLwqeIy)x7PdRe6UMIImjO+0E|@yD&#DlLdPEAQ)pli@&9V&E2EUmTnLnsi8-@gh5I{0cjX|0BJ^2x*3re%0Z+=N+gs4kscZeK|mN9=@O)QFTel) z%lrLZYd-9C_u1#{6L;=j`#k#@zm4avVgyS_FhqT7XHKPz@(j&g)0M&+G3$ zd;Y8<^HDCDgY~P&O|wjv_3xqS{djgJZjq0S??)L66)<838KES%SsB&9^USg-Vl|2= zAGDJ-`!K>$!N2=o_Wmy4^BSyA3ijQA*zn5rh29IXojtIJOd@J{i6wH^OhJrwtFfxf zs|^%`#`m~|W~$UV^I(Xpiq2xY63QZ!%!P~4MX~*DNnvoCV`=3<**27v^nKp|_ zN6ULXAgCGx3pM}4(VQaPpL7NP_f0>yq7k>GEOMPPZvkOipQ+|0ta4u z?UBMnmS<#NKiIUB$j3zejxn&EmJ>+feqQuB9k{v22dXQ@LyyYY_L>_}IZ~#F&W~;M z9g}RGoji8*1)~LsuiIdyewkSvaCC7{w?OCEr^yFUnDcI=^Rm z)+$Sc^TIkAi?k58!1fiK?4s6bqa758dI=gTDVqX0eEn@F{MtV1o7Rxgk!hv|F4@P% zQ*sK9oD;=UQ>R{xHJx+_^5iG0(FHW`jHPg`{YeVDHNml~yrDL4(+nvoj@&w{?alKc z&8`*W{r-`ju9r|uhg7`-xO9~Q>e?LrchToNVT6wKpC%+A5Qw`?4)?o%KX^jVmqV`> z_V4`mU7$f#$WhwH!3#NGJ*kiBk|v@i9%jAR2gUtD*kG#I2tO_*?P6W*is*m?t&RpB-?gN7hh6oPnCti>K1vA$E@n7^|T)?WQ@+3enW!_G4s~cW)t1!EU)nI%i@ynI9Ved7*N((>#l(+D+?nntT=%iP7Kf8M!=x278 zsvrCN`Of&`wa%rJUj9%m;;XZxlU$jFDr;n|Bc+FyRvrAAMoy-roXW-Wc_&U;x4*_@ z&mi=bvI2=MSbS89DvF2#(%lu4NkhE}Vcw)8pz2Ocg`4YzE(!MLhk+KR)>PL7ej{vQ z9T&g8U%T&ozkW!?{xy1HVyar16$a2g9QsqN^FFpQmIQ;r4mM^mNb|q%CueBn$zL-R z{0mj;rZVZoViZryk_J3Y*G<svYGnY>;}gS_m( zZBdXA5%r+u2pcV=a(riI}^$uGmt`p;lgasGFm3&KtkNm0_^dh~bAdB$v#xjRNY2 zSK%#p>*(5bVLK)(&8;z}`O4W><}ifP$^31rkfRU)sCveN%3+2<`RT|8G-aivG?eni zIJKm~;8q|R4+HYa)`yW?7q7&E7s$!4SlfTR?oB6l`Fpy&m#iiy95naG`EhXQ!g;YZ z$|`M8Gi=qHXCJ}vZba+!-dCE2Cn~6zS}k5$A#z-wjo9x1-mm17r+208c_rY5d|$aZ zutA$NL*~_X%1Yr66q%z-HtoE?GD*iYEMRmb3s*=?LEDK^(Tr!4lIlgX4)kpRX-z#_ ziKQAD@aNW2gW9Z;PuvX+z$RH$;nivrsHzC0f#9eW2vtZ(h`j`h%!|3^aWOAdW>~*w zV{zg9=w21YnlpnX>#A*+bKoMzt>Nz#J&&c_v`A83xt@ao!QbV15sr8hC1a&NW2jCV zIo`>MI^y5G<78zU4APePT->x`(=LK$$&}&=>4&4>Bx&YhK412VB{j^Q7nV%%}Zav;r5$mP<0chL#8L?H4Kd#2!3vtK5wWf#PToBo|qqX)&ZrHs)Hm)<^5_ z0x?NJ3u(<3kKwX)<2L2GR75E!T-(e~nx)>AS6Z78`Kws zQI)?;4>)=!YWZE1`tsmt-ClxKdvo_84BRMeYap9lSWVx2O4f)}=4EDUvm;Gzo3{%= zf>C^(nju~=?WA59JCopuTC)$IJJVL{OkGAb?v(B7)Vd0&q#r^bugTbm={~j@%8M{! z)uzb;srw(01Rad~uTT4)e!v84SYl)>e=S&w3d8OxRKD*e>_d{E}Sy@wWJ zBZvx~-Q!?SzOH-2PIZVdPIpt<1GI z{^A;4+u)vc<^%$xm}1S(f@uWF@;`QHlWrVt5an@8tbC7uZ^mTof%CKe`)7l z4nh3h8+y!kecZ16##ztgTNwUvI|1S9?`PygMAHHyqYPy7&84~1z${5Tx8r|E^)SxAd>tU)s|5T^i5vtvT4&4$~LKK%P9TNMo%G$tEUQygojQx>K4_v0L9db)1yfrD+H?iv50s8BPlF`;6`0K1?AQH_X@xRA_zUDZNZkZ~%h zaCP&+Pwl39womm;y0lpvg3`n(OY`SS=rvk`@CYM~RXwR>7LM$xks8QYb~MLa#?hby ze1PwHOi&QZw(`TdlVv)gDLMp%YWeV`1rw&}hF_@kSEG-hiG^3!`{v`W$Fku!tVeS4 z@{NZ>@~E@obbxB59{{IneVvaWF9}Xrf&lb$!8jAo1Zamhdn`g>I{x>Z(yXK)>doKd zncu$0+~gz{hE(j{{(_@wH3&VR2f(!wbnygV zX8p|v|E=%tuEnr|!>aNakd>Yz(6PJcCoMSVTK8wT51Wg!h~p_IfR{ih(He(XN(O3f zkAN&HW;G}&Dd|(dt`)EdoMz_crx$C=!}XI^{Yyw?rP9qez5Me2Tl1%D`T6_1&mJsB zIp4mZxQ5cwy4;JVd>uVihfvU$ds|XmTs(4SqhBr>n%EIbT?>@PKbT$mGg62^V<#%j z6Orx!qz3SI;1mPw7g9ZzYVfQ+gUG3XQ5!gD$rWbMhPpZ_OK9fDL_KAEeEhol!REw( zvawQwF{_$KvvYGpdwY9%)o?hxu(Z@kznZ#aj=iYNjMjx9x@@A|mS7*z44K8S&vk6@ zf=#L{UUu8arvWP+6BEj^^@!K?x;GPGLfT)r`z$Yo`aZ@H`@#QVYDK!344o)Vh00~!=Au78Cu7<$@KS&eINK~ zVy$@n{RBc-8Yf)e_uAJIWiO!(hYOWGb)AZ_FT;=wREKQCfy&qK_HJ$zuK?g5>goNi zf8Vb|07MI^VHpE}GkdXrIy*bN^}Eq$Gz5spL%X}XfonyMCjaflQsqmRDX5psBrr8( zc@-w1s;cVrs$TGkXF`wNecdmtumqlLD1U;y1rR5GrtUTLdlE~!_iZ->D1x7OyI+)V zA&LcPBVTUUPdfU_EVxyl|6WtVs#7a=Ml)HN9!ekHl`8@07V}LwO5+A{)u4D^tIU5J zjU>sNJ88#O%iag3HSmmwpZr^}4mHD%^Im=+#44R>_j+Z?C z_cZWU#02DF(bM&uqIyQ$KkZ_}qA?e`_|Es~ky~@$a=9-5Xpx^uIxZdbizzSJ%8Wg+ zk>LjVPybjd@w(5EwbnhMhDHzKII3HigMoZiaV(|Dt1+#BJS)ei(a4M)T{pfpe3)QV zrE~i4dPIwBB?n%`lkw=bC%~OagT$_HZ;Qx$`PC9~&H`UErlp4E5=x9_%G1R`$L#ChF zJprBl|N3q>5noA4yX6(lZ!cf6m4ApO2>Hgt}E;Y~?5hhZh z6NonPkAc^^MmVy{=@^%ou;!_F1)$>wk}%;?3WO!&nr$EnpADkXrwmyVl7P4CWo*zf zLz}*>dyt0$>1Z4o@D->r)#aJ802;`W=|8W!Pc0`FWncIc`e+b%qznWz-VY8=72eIK zuRg7wRk43^59|;Mf)q~)PfE~aYlMTy)!RVH`VQQNs(`Em$QlmZ5|o4`!c;Jd(_pJM zke?Qy4A0*j{se6M6Lgg479^rA?9XKGioc7ph=}8(@ixb)6WQx(*+jI zaL}9lM$Q8!WG`pS%s2_j6@fe>;+5b7+L=k|cz&eNu%BSa;*}hU`_KMmK`TRcvL?xm z)qoITNsq<2`G1Yt?tve!b{G$_G4TH3O3Jfr_JMOe&k`Q7HY zI1*Odn_+>o$IEMoPgoKc2c-n2!OJWC_&I=6O&5b{c6};3` at the chain on $S = \\{0, 1,\n", + "2\\}$ with intensity matrix" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "Q = ((-3, 2, 1),\n", + " (3, -5, 2),\n", + " (4, 6, -10))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The following figure was shown before, except that now there is a black dot\n", + "that the three trajectories seem to be converging to.\n", + "\n", + "(Recall that, in the color scheme, trajectories cool as time evolves.)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "tags": [ + "hide-input" + ] + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAAFUCAYAAACp7gyoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxU9b3/8df3nDMz2QMEEFAEAUVFReuGigsiisCtgoqIgoAKKirS4gJSBJVFrxtg66+ttlr9aX/3arW9VxBQwYXWpe51R6GA7IGQZbazfH9/nJkwxABZJjlzZr7Px2MMJpmZTzI55z3f9QgpJYqiKIqi7J/mdQGKoiiK4gcqMBVFURSlAVRgKoqiKEoDqMBUFEVRlAZQgakoiqIoDaACU1EURVEawDjA19WaE0VRFCXXiPo+qVqYiqIoitIAKjAVRVEUpQFUYCqKz0yfPp1HH33U6zJ+4pRTTuGLL77wugxFaTEqMBXFR7Zv386f/vQnJk2aBMDOnTsZPnw4hYWFdOvWjeeee65Bj9PU+z322GOcdNJJhEIhxo0bt9fXpk2bxqxZsxr18yiKnxxo0o+iKBnkqaeeYsiQIeTn5wMwefJkgsEgW7du5ZNPPmHo0KH07duXPn367Pdxmnq/Ll26MHPmTJYtW0YkEtnraz//+c+5/vrr2bx5M507d27eD6ooGUi1MBXFR5YuXcrZZ58NQE1NDS+++CL33nsvRUVF9O/fn5///Oc888wz+32Mpt4PYMSIEVx88cWUlZX95Gt5eXmceOKJLF++vGk/nKJkOBWYiuIjn3/+Ob179wbg22+/Rdd1jjjiiNqv9+3b94DjiE29X0McddRRfPrpp81+HEXJRCowFcVHKioqKC4uBqC6uprS0tK9vl5aWkpVVdV+H6Op92uI4uJiKioqmv04ipKJVGAqio+0bdu2NtiKioqorKzc6+uVlZW1gbovTb1fQ1RVVdGmTZtmP46iZCIVmIriI8cddxzffvstAEcccQSWZfHdd9/Vfv3TTz894MSdpt6vIb766iv69u3b7MdRlEykAlNRfGTIkCG8+eabABQWFjJixAhmzZpFTU0Nq1ev5q9//Stjxoyp/f5x48b9ZPlHU+8HYFkW0WgU27axbZtoNIplWQDEYjE+/PBDBg0alP4fXFEygApMRfGRsWPHsmTJktolHb/5zW+IRCJ07NiRK664gscff3yvluKGDRs444wzfvI4Tb3ffffdR35+PgsWLODZZ58lPz+f++67D4C//e1vnHPOOXTp0iXdP7aiZAQh5X73V1ebrytKhpkxYwYdO3bk1ltv3e/3xeNx+vbty2effUYgEGjw4zf1fqeeeipPPvkkxxxzTIPvoygZqt7N11VgKoqiKMre1NVKFEVRFKWpVGAqiqIoSgOowFQURVGUBlCBqSg5QEqJ4zhel6EovqauVqIoWSgZkJZl1d40TSMYDKLrOrquI0S98xoURdkHNUtWUbKE4zjYto1pmpimSeqxrWkahmGgaVrt53VdxzAMhBAqPBVlb2pZiaJkEykltm1jWRbxeHyvLldN034SgkIIPv/8c3r16kVxcXFtcAohaludmqZGaRSFfQSm6pJVFJ9IDUjTNLFtu/ZrQoh6Q3Lt2rWsXLmSVatWsWbNGvr27cusWbMoKSmp/V4p5V7dtqrLVlHqp1qYipKh6o5DmqZZ+/lkOO4v1EzTZPz48Zxxxhmce+65HH300ei6vt/nS/2Y2upU4ankGNUlqyiZbl/jkMlwbEpwxSorCVeG2bTd4YRTuzfoPlJK1WWr5DIVmIqSaVK7Q03TPOA4ZGNVbtjAti++5M5Hd/DmBzW89tcBnHxu4y7jlRqeqstWyREqMBXFawcah0znjFU7HmfDW2/x0DO7+ejLGGedehAr3trAP96/isKSgibVnvpRddkqWUztJasorS0ZkLFYjOrqanbv3k11dTWRSATHcfZqsaU7eBzL4qn/3sIbqyv45WXtOPdIm3bFBrPufrdJj5cM82Sdtm0Tj8eJxWK1reMDvAFXFF9TgakoaeY4DvF4nHA4TGVlJVVVVUQiEWzbrg3I5JrIlmyZ/e+rm3j6lQjTr+pISZGBY9lcN6SUF15cy2uvrW/WYyeDMzmmaVkWsViMeDyOZVkqOJWspLpkFaWZ6huHlFLuc6lHa/j73zdz5ZXL+dXEbnQwt2BGoxihEAXt2rGp4HDuu//v/POfV9C+fX7anlN12SpZRI1hKko6tOY4ZFN8880uLrzwf7j33vPoam+gZttWAoWFYNuYkQjdBwzg8T99w/btu3jhhaEtUuu+Ztl6/btRlAZSY5iK0hRejkM21ubNNQwfvoQpU06nX7+uBEtKcGIxqtdvILJ1K5qm4ZgWN998Gj/8UMWTT37RInWoLlslG6mdfhSljmTrKLWbNbW15HUo7ktlZZwRI5YyfHgfhg3r7X5SEwjDoLjrIQhHYtbUYIVrKAoaLFhwIePG/TdnntmF3r3btVhdyVZl8veaXGOaKW8yFKWhVAtTUXAn6pimSTgcpqqqisrKSmpqaojH4z9ZuJ+JJ/d43Oaqq1Zw9NGdGD/+Z7WfDxUWIqSkesMGajZvBiSOZQHQo0c7Jk8+jbFjlxOP2/t45PRRs2wVv1OBqeSkZAsyGo3+JCBhz9U9/LBAX0rJTTe9BRjccceZe9UrdB0BFB98MEWdO6MJgVkTrv36yJHH0q5dEbNnN22pSVPtq8s2FoupLlslY6kuWSUnpO7LapomVqKVBZndzdoQ99zzAV99tZvHH/85ur73e2CjoAChadT8uAlN1wgUFiEts/brQghmzz6PkSOfY9CgQxkwoGtrl79Xly1Q22Wbekkyv742SnZRgalkpdTxsuRYpB/GIRvriSe+5IUXfuCPfxxBfn7gJ18XAFJS0OkgNNzdf6xwGHcCvPvzt2tXwJw5g7jmmtf44INRlJWlb6lJYyRfj2R4Jtezqlm2SqZQXbJK1qg7Drl7927C4TCmafpiHLKxlixZx4IFH7J48TDatq0/5PRgkEAoj9i2bUR3lINpgmlBnS7PM87oxnnn9eLGG1dmRHeo6rJVMpEKTMW3DjQOmdxRxw/jkI31/vtbmTz5TR5++EK6di3d9zfqOo5lEmzXjlBpCWgajmXhWD+d5HPLLWfwzTe7eeqpL1uw8sar2yNgmibRaJRYLIZt2yo8lVajumQV30h20yXHuBpyAeVstGbNbkaPXs7ddw+kT5+D9vu9mqZj5OcT27YNR9PQgiGkbSEdG9i7CzcUMpg/fzATJrxA//5dOPzwti34UzRe3S5bKWXtm6PUN0a58DegeEO1MJWMlbphQE1NDZWVlVRWVmbkhgGtZfv2CMOHL+H660/hzDO7HfD7RcAA20EvKEALBMC2cByndmlJXb16lXHDDf0YO3Y5ptnyS02aqu4SldQu29R1s4qSTiowlYySOg6ZGpDJcUjDMHJ25mRNjcmlly7lgguOYPjwoxt0HyEERl4IGYujAXooBKaJNOsPTIBRo46jpKSAOXPeS1PlLatu70Kym1512SrppgJT8VTqOGQyIFM3DEh2tWXjOGRjWJbD2LGv0b17Gddff3KD7yc0DWEEEJrAicexq6qQtp3okt3HfRJLTZ5++iveeuvHdJTfKuq2OpNdttFoVG2MoKSFCkylVdWdqJPclzUajSKl3KubNZcDMpWUkltvfZtIBKZPP7vRvxctYIAQGPkFBNu0RUhworH93qd9+0Jmzz6PCRNWsGtXtDnle0J12SotQQWm0qLqG4fM1I3LM9WCBR/xz3+Wc//9FxAI6I2+v15QAKaJVVVJfMd2pGUipXPA+5111mGcfXYPJk/OjKUmTVXfEhXVZas0hQpMJe3qjkMmL6BsWVbteshcHYdsrGee+ZpnnvmGhQuHUlDw040JGkIXGnogQKhtW0IdOqAZBlZNTYPue+ut/fn88508++zXTXruTKO6bJXmUMtKlGar7wLKSanv7JXGWbFiPXff/T6///3FtG9f0OTHMQrykaZFvLocIUALhhp8pdu8PHepycSJf+H00zvTs2ebJteRSeouUUn+/dbd4EJRUqnAVBqtvgsoSyl/Mm6kNN3HH29n4sSVPPTQELp3b956SKHpaIZBoLQEpMQxTeya6gbfv3fvDlx33SlcffVyVq68pEndwpksde1maniqvWyVutRbKOWAGnIB5eRsVnVyab516yq5/PJXueuuc+jbt1OzH08vLADHJr5tO2Z5OU4kgmMfeAwz1ejRx5OXF2Lu3A+aXU8mU122yv6oFqZSr+SVPfx0AeVsUF4eZfjwJYwbdyIDBvRIz4NKiR4IoOXlge0gkTjhho1hJmma4N57z2fkyOc477yu9O9/cHpqy1Cqy1apj3rFFYDaK3tEIpHa9ZDhcNg3F1DOBpGIxciRr3LWWT24/PJj0/a4el4+OA727t3Y4TDCsiCxpVxjtG9fyKxZAxk/fgUVFftflpJN1EbwSpIKzBxV33rImpoaYrGYWg/pAdt2GD/+dTp2LOGmm/ql98GFeyFpLS8PTdeQtg22DU7jumUBzjmnB2ec0Z2bb16Vk0FRt8s2uRF8PB5XXbY5QAVmjmjIOKRqRXpDSsltt/2dnTst7r77XDQtvb97zTDQhHuoC11Hz8vDicWRDZ0qW8cvfnEmH3+8neef/yadZfpK3QluyWMruTGC04Q3I0rmU2OYWSp5NQc1Dpn5Hn30U95+ewu///3FLTMDVdMRAQNRuRs0HTsSwSgqQloWItj458vPDzB//oVMmvQXTjutM4cdtp/Li+WA/c2yVb002UW1MLNI3Qso192XVbUgM89//dd3/Pa3X7Bo0VCKi0Mt8hzC0N0rlpSUouWFCLZtix2NIhs5UzbVkUd24JprTubqq5djWao1laS6bLObCkwfO9AFlFOXe6iAzDyrVm3kjjv+waJFw+jYsajFnkcIDT0vhF2xCyccwdy5C00I2M8G7A1x1VUnYBgB5s/P7qUmTaG6bLOT6pL1keQFlJNdrFbKNQ1VN6u//Otf5Ywf/zoLFlxAz57tWvbJDB1sG6OkBGJx9/9NE7mPa2I2VHKpyahRz3PeeYdy2mmd01RwdlFdttlDtTAzWDIgUzcuT+7Latu2mqjjUxs3VnPJJUu5/fazOOmk1ljPKNCCQZyaGhzLRMbiCAnSav4Fojt2LOKuu85l3LjlVFY2fqlKrtlfl63aCD7zqcDMMPWNQ4bD4doLKKuA9LeKihjDhy9h9Oi+nH9+r1Z5TiFA0w00XUfXNDTHQZomDd5Q9gAGDuzJKaccypQpq9LyeLmgvi7beDy+V5etCs/MowLTY2ocMnfEYjajRi3j5JO7ctVVx7fukwcMhG6ApiFCITRNQ0bTd53LadPO4r33tvHnP3+btsfMFfvaGCEej6uNETKMGsNsZWocMjc5juS6696gqKiAqVPPaPXnN0J5WJGwG5ROHAyDdJ6HCwoCzJ8/mBtueIl+/TrRvXtJ+h48hyRbnsllYaZpYpqm6lnKECowW1hqQCZvaj1k7pk58102bIjw61//R9o3JmgQTaDnF4BlIgyB1DSIpa+FCXD00R0ZN+5Exo9fzooVIzAM1YHVVHX3srVtG9u29xqWSZ1MpLQO9RfdAupeQLmyspJIJKLGIXPUb37zOUuXbuChhy4kFPLmPaoIBJGxKNgOWDbCtnHM5s2Src/VV5+IlDoPPPBh2h87V6ku28yhAjMNUschkwFZ34YBahwy97z00g88+uinLF48jNLSPM/qEMEAAoEQ7iQgbBuauaykPsmlJr/5zae8//6WtD9+rlOzbL2lArMJ6tu4vLq6mmg0Wu/G5Sokc9Pf/76ZX/zibR59dCidOxd7WovQNETAcCfGSrfFSbxlrjjSqVMxM2YM4Oqrl1NVpZaatAQ1y9YbKjAbIHXj8uR6SLVxubI/X3+9i6uuWsF99w2id+/2XpeDFgwh4nGEdBBIiEWRLbjbzKBBh/Oznx3M1KlvtdhzKC51+bHWowJzH+qOQyY3DEi9iKxhGCoglZ/YvLmGESOWMGXK6fTr19XrclyaBoaBJgQaoGs6xFr2mpa3334277yzmf/+7+9a9HmUPepOJEx22cZiMdVlmwZqlmxCshVpWVbtRslJqe/eFGV/KivjjBixlOHD+zBsWG+vy6klDANNgnQchJQQCLbIGGaqgoIg8+dfwE03/Y1TT+3EoYd62y2dS+rOsnUc5ydzKtRwUePlbAo0dhxSUQ4kHre58srl9OnTifHjf+Z1OXsRgQAiMXygGQYiHkvL1ngHcswxnbjqqhOYMGEFdjOujqI0neqyTZ+cCUw1Dqm0JCklkye/iaYFuf32MzPu70cg0YIBhCMhHkcEg4gWmvRT1/jxJxKPw4MPftQqz6fsm+qybZ6s7pJN3TCg7gWUk384ipIO99zzAV9/Xcnjj/8cXc/Av6tAwO2OxW1tysRazNag6xpz517AqFHPM3BgV0466aBWeV5l31SXbdNk4JGdPuFwmHA4rC6grLSoJ574khde+IFHHhlCfn7A63LqJTQNXQg0pPtmUTruWkzZOt2knToVc+ed53D11cuprlZLTTLJ/rpsUxsaSpYHppSyNihVQCot4ZVX1rFgwYc89tgw2rbN97qcfZMSoenux3DY3Yi9hSf91DV48BEcd1xnfvnLt1v1eZWGq9tlm5znobpsXVkdmKrLVWlJ77+/lcmT3+Thh4dwyCGlXpezX8IwENJBkxKRl4ewTDQpW3QtZn3uuOMcVq36kZdeWtOqz6s0Tt2NEaSUxONxotFoTm+MkNWJknyhFSXdvvuugiuuWMbs2QPp06ej1+UcWGJPPAHuTQhwHHcSUCsqLAwyb95gbr55FRs3VrfqcytNUzc8c7nLNusDU1HSbdu2MCNGLOXGG0/lzDO7eV1OwwiBpunuP8M1CADbRrbSGGaq447rxBVXHM/48StwWjmwlebJ9S5bFZiK0gjV1SaXXvoqgwcfwcUXH+11OQ2n6wgtsfl6QYE7likltHKXbNI115xEOOzwyCMfe/L8SvPkapdtVgempmlZ+aIp3rAsh6uvfo3DDitj0qSTvS6ncYSGEO4BLwRoQiI8aF0m6brGvHkX8PDDH/Hxx9s8q0Npvlzqss3qwATVylTSQ0rJlClvE4nA9Oln++/vSpA42h2IhBPLSZxWnymbqkuXEm6//WzGjl1GTY3pWR1K+tS3RCWbumyzOjB9d1JTMtaCBR/x0Ufl3H//BQQCutflNJ6moWkCTYAWCrr/loDHY4hDhvTmyCMP4rbb1FKTbJONXbYqMBXlAP70p6959tlvWbhwKAUFmbkxwYGJRNcZCEN3Pzp2RpwBZswYwIoVG/jb3773uhSlBRyoy9bxaBy9KTLgcFGUzLV8+Xpmz36fxYuHUVZW4HU5TZac8KMJ0OIxdwxTOIhW2ID9QIqKQsybdwGTJ69i0ya11CSb7W9XIbuVtmpsjqwOTNXCVJrj44+3M3HiSh588EK6dWvjdTnNIwRCEyAlIMGxcQ+PzHh3f/zxXRg58liuueY1tdQkR6S2OpP7fmc6FZiKUo916yq5/PJXmTnzHPr27eR1OWkhNIHQQOSFEIaOlmFdYddeewoVFSaLFn3idSlKK0p22frhfK0CU1HqKC+PMHz4EsaPP4kBA3p4XU56JK9OgURYFrVHhp05rTnD0Jg3bzAPPPBPPv10u9flKK0oue93psvqwEzy42wsxRvhsMnIkcs4++wejBx5jNflpJFwl5ZICWYMrDhSAE5mjRsdckgp06adzZgxywiH1VITJbNkdWD64R2Lkjls22HChDfo2LGEyZP7eV1O+iV3LggYYBjuLNkMPESGDevN4Yd34M47V3tdiqLsJasDE1RoKg0jpWTatNXs3Glx993nomlZ+HcjBCAhFABdkCkTfuoSQjBjxrm88so6XnllrdflKK3ED1eXyvwKm0kFptIQjzzyKe+8s5UHHvDpxgQHkrxMCRJiMffi0RqtdgHpxiopCTF37gVcf/0bbNlS43U5igLkSGCqMUxlf/7f//uO3/3uCxYtGkpxccjrclqOBiBBOCAt3KMic4+NE088mBEjjlFLTZSMkROBqSj7smrVRu688x8sWjSMjh2LvC6nZQkSOxcI0DVEhnbJppo06RR27Ijx619/6nUpSgtSy0oyhGphKvvy+efljB//OgsWnE/Pnu28LqflCQGaBCMIuk4mty6TAgGd+fMHM3/+B3z++Q6vy1FaiF/O0TkRmIpS14YNVVx66VJuv/0sTjrpYK/LaSWJXX6sGDiWO/EnwzYvqE/Xrm2YOvVMxoxZRiSS+bvBKNkr6wPTDzOvlNa1a1eMESOWMnr08Zx/fi+vy2lFicBMXuYLCT45Pi666Ci6dy9jxgy11CRb+aFx44+jpZn80txXWl40ajFq1DJOPrkrV13V1+tyWpl0u2VrZ8YmAtQHhBDMnHkuf/3rWl59dZ3X5Sg5KusDM7m5r6I4jmTixJUUFxcwdeoZXpfT+pLLSgIBMPTERuz+UVqax333nc+kSW+wdWvY63KUNPFTgybrA1NN+lGS7rrrXTZujHLPPQOzc2OCA5EOCAnSZk/XrL+cfPIh/PznR3Hdda+p4zqLqFmyGcQPL4TSsn7968949dUNPPTQYEIhw+tyWp2UIEQiYKTlTvrRICP3xjuAG27ox+bNER5//DOvS1FyTNYHpgpL5S9/+Z5HH/2MxYuHUVKS53U5nhDJjdeFxL3Gl3A3X/dhSzu51OTee9/nyy/LvS5HSQO/nKdVYCpZbfXqTfzyl++wcOFQOncu9roc70gA212HKdzdfoQPW5dJ3bq15dZbz2DMmGVEo2qpidI6VGAqWevrr3cxZsxr3HffIHr3bu91OR5LzIgVgI67BlM4tdfJ9KPhw/tw8MFtuOuuf3hditJMfjlPZ31gKrlp8+YaRoxYwpQpp9OvX1evy/GeTC4hSQlOTfo6MIUQzJp1Hn/5yxpWrPi31+UoTeSXi0dDDgSmX14IJX0qK+MMH76E4cP7MGxYb6/LyRy1u/o4iRmzmq8DE9ylJvfeez7XXfc627dHvC5HyXJZH5hJagp6bojHbUaPXs4xx3Rm/PifeV1O5pAScPZ0wwrcST/C/6eAU0/typAhRzJx4uvqOFdalP+PlgPwy/oepfmklNx445voepDbbz9Tve57SQYmidBkz9VLssBNN53G+vXV/P73//K6FKUJ/LKFqT+qbCZ14swNc+a8zzffVDJ37nnoek78aTeck9isIBmaOAhN+L5LNsldanIhs2e/y9df7/S6HCVL5cRZRe32k/1+//svePHFtTzyyBDy8wNel5N5pMRdVpIISU0kWpnZcwo47LC23Hzz6YwZs4xYzPa6HCULZc/Rsh+qhZndXnllHfff/xGPPTaMtm3zvS4nM0kHt3UpQXPcpSWa8M3VShrqkkuOoWPHEn71K7XUxE/8co7OrqNlH/zyYiiN9957W5g8+U0efngIhxxS6nU5mcuxASdlKYlMBKbudWVpJYTg7rvP47/+61tef3291+UoWSZnAlN1yWaf776rYPTo5cyePZA+fTp6XU5mkzZgu92w0k60NEVWdckmtW2bz5w5g7j22tcoL1dLTZT0yb6jpR5+mYGlNNy2bWFGjFjKjTeeyplndvO6nMznJLtkSRz1iZZmlh4bp5/ejQsu6M2kSW+oN8s+4JdewOw8WpSsVl1tcsklSxk8+Aguvvhor8vxB8dOGccksUVedrYwk26++TS+/76SP/zhC69LUbJE9h4tKTRNU+8ys4RlOYwdu4IePTowadLJXpfjH46FO26Z3HydxE4/2TWGmSoYNJg/fzC/+tU/+PbbXV6Xo9TDb+flnAhM8E+TX9k3KSW33PI2sZhg+vSz1GvaCMKxQCTWYiauVoJuZG2XbFLPnmXceONpjB27nHhcLTXJRH7aXCa7j5YEv7wYyv7Nn/8hH39czoIFFxAIZG/LqEU4Fu4sWRI3CYHcWK86cuSxtGlTwJw573pdiuJzKjAVX3j66a959tlvWbhwKAUFuXGiTydhm7izZBMtTA0IGF6X1SqEEMyePYhnnvmGVas2el2O4mMqMJWMt3z5eubMeZ/HHvsPysoKvC7Hn+xYYg/ZPWOYUs+dNx5lZQXMmXMeEyasYOfOqNflKCn8dH7OicBU/Oujj7YzceJKHnzwQrp1a+N1Of5lxVMm++CGp5E7gQnQv393Bg7sxQ03qKUmmUQFZobx0wui7LF2bSWXX/4qM2eeQ9++nbwux9/sCLU7/Wi4oRnM87io1jdlyhl8/XUFTz/9ldelKPjr4tGQQ4Gp3lH6S3l5hOHDlzBhwkkMGNDD63L8z0zseCPAnSkLMhDysiJPhELuUpPp01ezZk2F1+UoPpMTgQkqNP0kHDa57LJXOeecHowceYzX5WQFYYfdTddFci0mEMi9FibA4Ye35/rrT2Xs2GWYplpqojRcTgSmn9b55Drbdpgw4Q06dWrDTTf187qc7GFHE8tJHHf8UgOM3L2yyxVX9KW4uIB77nnf61Jynp/OzTkRmOCvFyVXSSmZNm01O3dazJo1QL1maeS2MOWeFqYO5GCXbJK71OQ8nnrqS95++0evy8lpfjrOcyYwwX/bMOWahx/+hHfe2coDD6iNCdJKAk5sz/pLTXNvRu4GJkD79oXMnn0e48evYNcutdREObCcCUw/vYvJRc8//y1PPPEVixYNo7g4t0/kaSctcOJ7xi6FA4YGRtDryjx31lmHcdZZhzF58kr1htojfjo350xgqkt8Za6VKzcyY8a7LFw4lI4dC70uJ/vYJkLGU9Zg4l44Ws/NST91TZ16Jp9/vpNnn/3a61KUDJczKaJmyWamzz8vZ8KE11mw4Hx69mzndTnZyY65rUxNgu6ALhOBqVryAHl57lKTO+54h++/3+11OUoGy6nAVDLLhg1VXHrpUm6//SxOOulgr8vJXk4MhJnSJSvdK5Xk+Bhmqt69O3Dddadw9dVqqUlr89O5WQWm4oldu2IMH76U0aOP5/zze3ldTlYTVhghrMRFoxOhqWug5dbWeAcyevTx5MUWUYUAACAASURBVOWFmDv3A69LUTJUTgWm6pLNDNGoxahRyzjllEO56qq+XpeT9YQdRWK7k32SF5EO5rsXkFZqaZrgnnvO58knv2D16k1el5P1/Hg+zpkjRm1ekBkcRzJx4kqKiwuYOvV0r8vJDWYliMQYpuaAJpABNbmqPh06FPKrXw1k3LjlVFTEvC4n6/ntvJxTgal4b8aMf7BxY5R77hmIpqnXpFVYuxFCgi5AE25LM1DidVUZa8CAHpxxRnduuWWV16UoGUYFptJqHnvsM5Yv38hDDw0mFMqNixdnBLsy0bq0Ey1MkKFir6vKaL/4xZl8+OF2nnvuG69LUTJIzgSm4q0XX/yehQs/Y9GiYZSUqPV/rUlYFUhd4o5f4m6LF1ItzP3Jzw8wf/5gpk17i7Vr1VKTluK3hkzOBKbfXphssnr1JqZNe4eFC4fSubNq2bQqCUJWI7Tk+kvcGbKGeh0O5KijOjJhwsmMG7cCy3K8Licr+e28rAJTaVFffbWTMWNe4777BtG7d3uvy8k9TgzhhN2xS01zu2R1AwJFXlfmC2PGnICuGyxY8E+vS8k6frt4NKjAVFrQpk01jBixlFtvPZ1+/bp6XU5ussNAPLHxup2yLZ6aJdsQmia4997z+e1vP+fddzd7XY7isZwJzCQ/rv3xo8rKOCNGLOGSS45h6NDeXpeTu5xqBLFEd2zisl66AYYKzIbq2LGIu+4awNVXL6eyMu51OYqHciYwVQuz9cTjNqNHL+fYY7swbtwJXpeT0zRrN2iWu6RE1xJjmPmgqSuVNMbAgb045ZSuTJmyyutSsorfzss5E5jgvxfHj6SU3Hjjm+h6kNtu669+516zd4Juud2xuuPegsVql58mmDbtbN59dyt//vO3XpeSNfx2fsipo8ZvL44fzZ79Pt98U8ncueeh6zn155WZrO1IDXeyj7DdXX5C6qowTVFQ4C41+eUv3+Lf/670uhzFAzl1RlP7ybas3/3uC156aS2PPDKE/Hy1sXcmELIcodtud6yugS6RQTVbuan69DmIq6/+GePHr8C21VKT5vJbIybnAlNpGa+8so4HHviIxYuH0bZtvtflKOCuwXR2uke5ZiV2+dHAUIHZHFdffSK2rfHAAx96XYrSylRgKs323ntbmDz5TR5+eAiHHFLqdTlKklODptckApPE9ngBMNRr1ByaJrjvvvP59a8/5f33t3hdjtKKci4wVZdsen33XQWjRy9nzpyB9OnT0etylFTOLtCiYDhgkJgpa4De1uvKfK9Tp2JmzHCXmlRVqaUmTeW3RkxOBaam5dSP2+K2bg0zfPgSJk/uR//+3bwuR6lDk7uQWjyldemAEQJN7fKTDoMGHc4JJxzML37xltelKK0k5xJEtTDTo7ra5NJLlzJkyJFcdNFRXpej1Mfe5M6M1RP7yOoC9DagqSvFpMsdd5zNW29t4oUXvvO6FF/x63k4pwJT0zTfdQFkItO0GTt2BT16dGDixJO8LkfZByG2IPTk7j4CNAdpdPK6rKxSUBBkwYLBTJnyJhs2VHldjq/47eLRkGOBqcYwm09KyS23vE0sJpg+/Szf/cHnEiG2IA2J1B338l6GUIHZAo45phNXXnm8WmqSA3IqMMF/g8yZZt68D/nkk50sWHABgYDudTnKvsgI6LuQukTqAqlJpK6DfpDXlWWlCRNOIhaTPPTQR16XorSgnApMFZbN8/TTX/Pcc9+xcOFQCgrUxgSZrRxHj+Ho1N6kZoDewevCspKua8ydewELF37Chx9u9bocX/Dj+VgFptIgy5atZ86c91m8eBhlZQVel6McgGQzlm5i6QJLF0hN4GiFgFpS0lI6dy7hzjvPYezY5VRXq6UmB+LH83FOBabSNB99tJ1Jk1by4IMX0q1bG6/LURrA0Tdia2BrGo6GG5p6e3fjAqXFDB58BMce25lp0972upSM5seLR0OOBaYfXyCvrV1byeWXv8rMmQPo21dNGPELW9uILTQsIbAwcDQdRxzqdVk54c47z+GNNzby0ktrvC5FSTMVmMo+lZdHGD58CddccxIDBhzmdTlKAzmYWKIcW+hIoSOFhiV0EGpzidZQWBhk3rzB3HzzKjZurPa6HCWNciowk9TSkgMLh00uu+xVzjmnB5dddozX5SiNINmBLWLYaFgILCGQhNA42OvSckbfvp0ZNaovEyaswHHU+aY+fmzA5FRg+nGhrBds22H8+Dfo1KkNN93Uz+tylEYytY1YONho2BjYaDiyEIG6Sklruuaak6mutnnkkY+9LiUj+fFcnFOBCf58kVqTlJJf/nI1u3ZZzJo1QP2+fCgu1rktS3QsNCx0kGVohLwuLacYhsa8eYN56KEP+fjjbV6Xo6RBTgam6pLdt4cf/oTVq7fyn/85WG1M4EMSMMWWROvSvTkYCLp7XVpOOvjgEm6//WzGjl1GOGx6XU5G8eOb8ZwMTKV+zz//LU888RWLFg2jqCjodTlKE1jsJq5VYWJgoeNg4GCgO2rSlleGDj2SI488iNtue8frUpRmUoGpALBy5UZmzHiXhQuH0rFjodflKE1kik3EsbHRsdEx0XDII0Bnr0vLadOnD2DZsvX8z//84HUpSjPkZGCqLtm9ffbZDiZMeJ3777+Anj3beV2O0gw12lpMwEFPtDA1cEox1A4/niouDjFv3gXceONKNm1SS03An42XnAxMZY8NG6q49NJXueOOszjxxC5el6M0gwRqxI9YBBITftybLrsiUH/3XjvhhC5cdtmxXHPNa2qpiU+pwMxhu3bFGD58KVdeeTyDBvXyuhylmUwqiGo1WBiY6NgYSIIUyB5el6YkXHfdKVRUmCxe/InXpXjGzz18OReYmqb5+gVLl2jU4vLLX+WUUw7lqqv6el2OkgbV2kZikGhZJif9hAg5h3hdmpJgGBpz5w7m/vv/yaefbve6HM/4dU18zgUmqFam40iuu24lpaWFTJ16utflKGmyS6yrbVm6HzWEU0IAtWF+JunatZRp085izBi11MRvci4wcz0sAaZP/wc//hhlzpyBaJr6fWQDG4vd2nZMAsTRE+OYBvmymxq9zEDDhh1Jr14duPPO1V6XojSCCswc89hjn7FixUYeemgwoZDhdTlKmlSyhbBwEjv7uNvhSQKU2Gr8MhMJIbjrrnN55ZV1LFmy1utyWp1fz8M5F5i57MUXv2fRos9YtGgYJSV5XpejpNFWfR1xNEwCmIkxTGQ+RWr9ZcYqKQkxd+4FXH/9G2zZUuN1Oa1KBaZP+PWFaq533tnEtGnv8OijQ+ncudjrcpQ0cnDYKrYSJ5DY4ced8JMnO2Co/WMz2oknHszw4cdw7bW5s9TEz5MuVWDmgK++2snYsa8xd+759O6trliRbSoop1qzU8IygE2AMqen16UpDTBp0ils2xbjN7/5zOtSWo1fz8MqMLPcpk01jBixlFtvPZ1TT1XLC7LRv/V1xNAxMTAJuLNjCVHmdEVKieM4WJaFu7WBf9/dZ6tAQGf+/MHMm/c+//rXDq/LUfYj5wIzyc/dAg21e3eMESOWcMklxzB0aG+vy1FagIPDRq28tnVpomMSIN8uxbCCOI5T+7du23bi/x1UcGaWQw9tw9SpZzJmzHIiEcvrclqcXxsuOReYfn2hGiset7nyyhUce2wXxo07wetylBbyI1upFJI4AeIEEt2xBp3sbui6jmEYBAJBDCNQ+7fvOE7tTQVn5rjooqPo1q0tM2b83etSWpxfz8M5F5jg3xeroRxHcsMNq9D1ILfd1j/rf95cJaXkS209UQKYBBOTfnQ0QnTVuqPrBkKkHuICELWfk9KpbXWq4PSeEIKZMwfy179+z7Jl67wuR6lHzgZmNnfJzpnzPt9+W8Xcueeh6zn5Emel5HikbdtYlkWlE2GjEU9pXbrdsqVOW/IpOMCjucHpHgt1gzN7j41MV1qax333XcDEiW+wbVvY63KUOnLybJrNLa7f/e4LXnppLY88MoT8/IDX5SjNlBqStm0jpUQIgWEYfBXcRg0B4oSIEUhM+AlwuNO9Ec9Qf3CqVqd3Tj75EP7jP47i2mtfy9o39n49B+dkYGpadv7Y//u/a3nggY9YvHgYbdvme12O0kSpM1uTk3Y0TcMwjMQtgNTgU72CKAFiibFLE4MgIbrSoQnPuic43RpUd62XbryxH5s3R/g//+dzr0tRUmRncjRAtr1ze++9Ldx001s8/PAQDjmk1OtylEaSUmLbTspMVvaatKPrBpqm144/fsUudmAQJ49YYvzSJkA3pz0GzelZ2DPOqbprvRMI6MybN5h77nmPL78s97qctFMtTB/x64u1L999V8Ho0cuZM2cgffp09LocpYHckLRrA0kIaluSgUCgnkk7e7ym7yJCiBjB2tYlGPR1uqaxwvqDUy1LaR3du7dlypQzGDNmGdFo9i818YOcDMxs6pLdujXM8OFLmDy5H/37d/O6HGU/Urta97TaBJqm13a17i8kk76lkh+FQZQQEYLECWJhcJBTRBmFLVD53t21allK6xkxog9durRh5sx/eF1KWvi9Zy97kqMRsmWWbHW1yaWXLmXIkCO56KKjvC5HqUd9k3Y0TUvpbg2g6/oBQzLVX/QwNeQRI4RJkBgBJEFOdQ5qwZ8E1LKU1ieEYNasgbz44hpWrFjvdTlp4deLR0MOB6ZfX7Ak07QZM2YFPXp0YOLEk7wuR0nRkEk7qeORjfEFYb6XQaJOiKgTIi6D2BiUyCBHUNICP82+qHHO1tKmTT733ns+1133Gtu3R7wuJ6flbGD6mZSSm29+m3hcMH36Wb7/ebJBYyftNNXTlqQ6kkc0mk/cDGHaAWx0zrHbIjw5nNWylNZw6qldufDCI5k06fWs6B3zq5wMTL+bN+9DPv10JwsWXEAgoHtdTs5qzqSdplgVt1lbnY8VzseMhIhHA5h2gBInyMktMnbZGGpZSku7+ebTWLeumt///l9el9Isfn6Dn5OB6ecX7KmnvuK5575j4cKhFBSojQlaU7om7TSFacNTOwNQHYCwDlEDJx7EtnQudPLRyJQ3TgdalqI0VfKqJnff/S5ff73T63JykgpMH1m2bD333PMBixcPo6zsQFufKenQEpN2muK326Bml4BqoEZAVEBcp8zROIu8Fn3upqsvOG21LKUZevRox803n86YMcuIxWyvy2m05E5VfqUC0yc+/HAbkyat5MEHL6RbtzZel5PVWnLSTlN8Xw1vbtShEqgBokAMsGGMYXg0dtkYqcEp1bKUZrr00mPo2LGEWbP8udTEj+ffpEw/0hTghx92M2rUMmbOHEDfvp28LicrtdaknaaY/5kGFbity2rcwIxDN01yiq/GsJPBqZalNIcQgrvvPo8///lbXn89O5aa+EVOBqaf3uHs2BFhxIilXHPNSQwYcJjX5WSV1p600xSPfgC7Nwu3dVmN27KMAQ7c0tbPIaOWpTRH27b53HPPIK699jXKy/211MRP59+6cjowM316djhsctllr3LOOT257LJjvC7H9+qbtCNE60zaaYp3/w2rP9dhF25ghhM3EwYU2xwa8ra+9FDb7zXV6ad34/zzj+D669/I+HNZKhWYPpTpL5plOYwb9zqdO7fhpptO9boc30qdtJM6HpnsbnVD0puu1v0pr4SHXtJhB7AbNygjQByKdcmkQ72tL/3U9ntNccstp7NmTSV//OOXXpeSEwyvC/BKJgemlJJp01ZTUWGzaNGAjK41E7khKZHSqf3daZqGpmkZF4z1sSyYslBzu2A1IATYuCs2iuD2Yxx0Pw1dNor7ermvm0y0OkEIDU0TtV9XXMGgwfz5gxk//gX69+/CEUe09bqkrJb5Z48Wksn7yT700Cf8/e9b+c//HKw2JmigupN24KeTdvwQlgA3zBTE1gooZ09XbASQcF53myPLvK2v9dS/LEWNc+6tZ88yJk/ux9ixy4nHM3+piZ8bAP44g7SATH3Rnn/+W5544ksWLhxGUVHQ63Iymh8m7TTW5FtsKr6XUOW4XbE1uIFpQ8diyaRTPC7QE3svS1Hb7/3UyJHH0aZNAXPmvOt1KVnNX2eTNMrEwHzjjY3MmPEuixYNo2NHr7c6yzz7mrSz93ik/0IyadKkcrZt3g5sx01K0/0QByMoeeASx9sCPaeWpeyLEILZswfxzDPfsGrVRq/L2a9MPPc2lD/PLGmQaV2yn322g2uueZ3777+Anj3beV1OxmjIpB2v1kemSywW48orP2Dnzm+A9eyZ6RMFJOhwzziHQvUeKoVallJXWVkBc+acxzXXrGDnzqjX5WQl/55lmimTLiK9fn0Vl176KnfccRYnntjF63I899NNBGRKV2swK0Iyqbo6yoQJK4nHN+EGZTnuLgWVJANzynibww/xsspMppalpOrfvzvnntuLG29cmVENAsj8ZXwN4f8zjs/t3BllxIilXHXV8Qwa1MvrcjyTDMnkdnTAXiHp567W/Vm8+GPi8RjJTcshDlQlPgquutqhf06OWzaWWpaSNGXKGXz55S7+9KevvC6lXqpL1ocyoYUZjVqMGrWMU045lCuv7Ot1Oa2uvkk7e29qnp0hmSSlZPPmatyg1HEPRwFYgMaoUflcNMy/Jxdv7LlaCuTmOGcoZLBgwWCmT1/NmjUVXpezFz+HJeRwYHr9wjmO5NprV1JaWsjUqad7WktryfZJO40lBLRpE0qsL0yezCWgM378QVxySaZehcQvcnec8/DD2zNp0qmMHbsM08z8pSZ+kRtnpgw0ffo/2LQpypw5AxMnzOyUC5N2mk4watSRdOiQh65LwCIQCHDXXT9jyJD2XheXReoPzmxvdV5xRV8KC/O59973vS6lltcNleYSBxiIzdq/JsuyqK6uRvdgy5TFiz/jj3/8miefHE5JSfa1IpI77aT++fhpp53WtnlzJd98s4tAQOPkkzsRDO77wuCRSJyPPvqRL77YTiwWIBQqpKioiLy8fPLy8igsDFBUbFBYAsWFUFgEhXkQUNcaT+H+XSbPfdm8i9COHTWMHPkczz03mDPPPNjTWpLXwgyFfLEJcr1/DDkbmLZtU1VV1eqB+cILa5gx413+8IcRdO5c3KrP3ZJ+uh2dQNNEToRkefkOXn75ZdasWUOvXr24+OKLKStLfwuxoiLMnDnL+Pvf/000auGOexYDHYEOwEFAW/dzIgilQDvcj21BbwNGCYTaQF4bSUEJFJVISougQxG0LYQ2+dDegHa6pL3m0B5JKQKdbN1xStYTnJBN4fnWW2uZP38l//znFbRp411YOY6DrusEg77YkEUFZiovAvPttzcxduwKfv3rn9O7t7+73KSUtbckIUTtSSfbQzKpvHwHv/zlNGKxKJZlYRgGoVAeDz30YNpD809/eo8nnvgn1dUWtg3u5KAA0B43MDsk/t0GKAJDc/OzJPGpUtx8LUn5WJS4lQCFe/5fFIJW6BDMi1FYGKVNKEpbYVJmm7Q3HTqb0NEJ0s7UaBMPUWbnUWiFCOLXpmxqcIraW7YE5/z5K4nFIjz77AWedYs6jlM7oc8H6v0lqc3XW8lXX+1k7NgVzJ17vm/Dcl8h6bYiMzsky8vDrFlTQSAg6NWrbdq6wl9++eXasAQSH6O8/PLLXHPNtWl5jqQtW6pwV9wkf88a7ntaO3FzEreU97nJ/3Vw89VJ+fbUu1gpNxOkBbYpiAYNTCtANGhRGbDYkueQT4y85E3GKJQx8qVJvrQpcixKTUFbK0CpmU9ZvJSieDEFVhEFVilBWYCWkVMnkgHp/u6SE9KSPSV+D86pU8/kiiue5//+36+56qqjvC7Ht1RgtoJNm2oYMWIpv/hFf0491V8r0P0Qkrur4YfNsPZHWL8WtqyDbZtsairDOM4O3K3mdgAmQkBZWT4TJx7LCSd0bvZzr1mzpjYskyzLYs2a75v92HUdcUQHDOOblA3IbdxD2MBtaaYuTUlI/m/qyhUt8e9U++iJlFKAFOAIHAQSHXD/bSOwhCAmNCQCBwtLxogEY+yUFiERIyTjhByLPGzypU2+DQVWHgVmCcWxduSbB5EXLyNolaHZbTDwursuO6+WkpfnLjWZOPEvnHZaF3r2LPW6JF/K2cBMSg5Et5Tdu2MMH76ESy45hiFDjmix50mnn07aEZ5N2rFt2GrD+hisr4FNVbBpp6B8pyBcIbCSm+JU4O4mtxuI27i75CQ/UQmYgIaUgp07Y/z5z19z+OHtKCpq3phOr169WLdu3V6haRgGvXr1bNbj1ufCC/uwdu1OXnzxC6qq4rghWcLefa1FQD6gQQGQl/jfPCCYuAXYO2OTQepm4Z6PAhAShAMauMN7DhIQyWAheQxJNAlSCJDJNZCJTQQExDUbXZo4mokZqCCav54qYRF0bEJYBGxBSAryrGICsQ4EYgcTMruimZ3QzE4IWUrrh5XYT3Amf3r/6N27A9deezLjxi3njTdGeHIlpExY/94cORuYe8YoWk48bjN69HKOO64L48ad0KLP1VzJ5R97tE5I2thUEmMrJlsx2SgkPwqdrVaAneEQ4UgQp1p3NyGvxr1yR1Xi39W4n48AMdzNcSxwg95M3JJ9j4LUVVTV1SbhsNXswLz44ot5++13gL3HMC+++OJmPW59QiGDqVMHMGHCqWzfXkM8LojHBaYpiEYDmGaAaCxAJK4TsxximvurCeP+aiIB99dVpUF1AGIBiOYLoiFBPA/MAnDywckDJwgyZGMELYJBC0Oz0ISNjo1I9OtqiTanEKBJiRSJMcDkG61kD2fio8QNXZF4LSTgCIklHXQjjilNpFGOnfcdNja2dAhJMJwAul2Abh6EHuuOjPdExrpjWweD05aWXx1XX3D6s7v2yitPYPXqfzNv3gfcfXc/r8vxnZwNTGjZblnHkdxwwyoCgRC33dY/49Yf1dfVuick09vVGiNKNVVUUEklMbZrYXYKiwqhUYVODSFiBIkQIirziFgBorEgjhmEuL4n+1LH4VKH65I5X/ujiDo3PeUb3RNfcXGA4uLmd/+VlbXnoYceTMyS/Z5evXq22CzZpNLSAkpLC9L0aHsv/4lbsFuDnZrDzrhDBVBtGVQHHCoMi4hhENFt4gEBQiI0iSEcDByCSAxhowsHDQddOonwdDCkG6TuSyYT/3K5h4YELdGgRbihLGyktJFaNRjlEPweregtAlJgODqGHUCLt8GJdCUeOxorfjSmfRim7ILbvE63vcc5/dhdq2mCe+89n8svf45Bgw7l9NPV3tWNkbOzZAEqKytrF9Kn26xZ77Fq1WYef/wi8vIy431JS41HOtjYVBNjFxF2US0qqBYVVIkoYeEQQSeGToQAcQLEEwEZI0iUECYhooSIESIm84hZQWKRPOxoEMK620RKtiSTrcowe1qZyRZnZeLflsOepuguYCdun20cgLKyPKZM+RlHHdWhST+vskcNMcJ6lGojQk2whrBRSThQRU2wHFurwg5UI7QwARFDF3EMESOASUCaBIijYxPCxBAmQWwCMoaOSQgLHYcAFkFpomNhCJuAtAhgE5A2Aoug5WDYNoYlETEDzEIcsxDT7ErUPoqYfTSmPByTbjhaaTKd06i+ZSmZH5wrV/7Agw++yQcfjKK0tHWWmkgpCQaDfumWVbNk62qpS3z97nf/4uWX1/KHP4zwPCyTAbln1l/TQlLiYLMbyQ5stmJqO4izlahWTYwYMSRxdOIYxDBwT4dB4gQwAQsdiUAmuuIkPz2xiMQzydr/4jY5NLFnskrq2FsAd0wunz0tTgHUaBDPx9B1CgoM2rQpoqysI5062fTpU0Tfvu3Jy/N6ckl2KCREoR2ig13qdovXI0aEqLGbWKACM7iTmL4VM7gFGdgBxi40vRpdOOia5XbvOhIEaA4I4bgtWSkQMjGeKhO9BbXHrkRKEJqJplWi6VUYzmby+Sc4Oo6Vh2OXYMpOxOwjiIoTiAd6YendcPTmhmj93bWZvixlwIAerF69jltueZOnnz6/VZ4zG65WktMtzJqaGizLSus7nv/5n7VMnfoOTz45nEMO8WYmWv2Tdhq4iYBdA852sLci+BEhfsTWt2IalVi6iSUEcXTM2lsQEwMr0Xo0Uz6atYG5JzyTH2MEE/8O1f47StBtbVpB4maIeDQE0QBEEoNxUfdmxCA/LilxoL3mcFAAuhRBtzL3Vpw9+0FkPRsLW9+NHdiBY2zFCqyH0Hr04I/o+nZ0rQpdRDFqW6cWeqKVqUuJYVkEbBvdBM22EZaOFhdIGzAF2AJp6mBJhJn4vKVj2/nYdikWnYjpRxMN9SUePAIrdAjSaM6FR/dez5nJ45yRiMkVVzzPXXedzOjRvVv8+RzHIS8vL+OGp/ZBbVxQVzgcJh6Pp23zgvfe28Llly9j8eJhHH10x7Q8ZkPVP2lnHyHp2GBWQHwrxLeixdcjnE0IykGvBiMKho00HOyggx3QsXQDUzOwMBIBaSRC0agNTLO2y3VPgMYJ1gnR1M/tCVGLEPHENE4pg+TZQUrsAO2sEB1sja7oHGxAB9UwzBmSGOg7EIFNiOA6jNB3GMEfCAR+JKBVYMhqDDtCwLIRDugmYGlggrAlWMKd7xUX7uctwAZpG2BJpKUhbMASSNvAsQqwRBtMvRvRguOIFh6HmdcdO3QQaE3pKcr87tqvvtrG9de/xOrVIznssJZ9g68C0+cikQixWCwtgfnddxUMHvw3Zs06l/79u6Whuv1L7WqF1HezKZN2bBMi5RDZiqjZjIhuRMS2IJxy0CKgx0GzIeAkujqdvZb1OUEHxxDYho6p69jCwBRuQNqJ7tdkaLoBGkwJ0CBx9MTHILY7GoVDCEEIQxZQIAspdkopJZ9S8mlHPkW+3SlGaT0WQtuOYfyboPE9efpXBLXvCDqb0Z1qNDsMpo1wJNLWwJRg6wjLSYSkTISk2wLFkmAHwHJwLA1hCbA1HCeIQyGWn/gOZgAAGbhJREFU1pFo0dFEi/sSLz4Sq6ArMtCYSUWZvf3eU099yOrV3/PaayMwjJYZX0yer1Rg+lg0GiUajTY7MLduDTNw4MtMmHASF13UcrtopE7aSa4fFUKg2RYisgtRvR2tchNUbUREtiDMXSAjoJlgOKA7YEjQJQRkYhww+dFJrM8TicCUOAFwAhLb0BItTB07EY5xt1OsttXoTs9wQ1GSD7IYQ5YSkh0IyTYU0oYCSskjDy1r9yVVvBXBYDMBsZag/IZ850sC9loMuxzNioBtIiyJtASa5eA4OsJxEl23mjtZzBJgG0jbcVulto60JcLWkFLDdgqxtbbEi3oQbdOXWLtjMYsPwwk1ZCw0M7ffcxzJ9de/xMCBXZg5s2WuVp48Z+Xn57fI47cAFZh1xeNxampqMIymT8ypqopz4YX/w+mnH8akSSensTrXXpN2LBtqKtArt6Pt3oK2+0dE5WZEtAJkFIi5LcZkMBq4YWiQ+Jxww9KQeybOGOz53kAySN3QlAEHGQBb17ANDVMLYGshLELYFINsC/IgNKczBh0xKCNIW3Sy7wosik85NpqznaD8N0Hra/LiXxKKf4tu7kBYYTTbBNNG2jqa5bitTkdD2E6iC1dHWhIsDcfREFJDmBLHEeDoSBHCFqXEi7sTbdeXaNkxmG174hSU7SdAM+9qKVu3VjNq1HO88MJQ+vVr/g5YdSV/1rw835wbVGDWFY/HCYfDTW5hmqbNyJHLKC0tYubMc9LW1SBtG6dqN2LXDkT5FrQdG9F2bUWrKUdYETccdScxjZBE+DmJWaSJYDTkntakIRI7ujjuhtyGkxKmyc8lvi8gwDDACCCNfKTRDml0wtY7I/UuSDohaIeOb94pKsrepINu7yAQ/zfB6DfkR74gUPMNRrwczaoBy3RD05Jul60twEkGqAa2dLtsbR0hBdIRia8JpMjD1kuJl3Ql2vEEYp2OJda2B7Kg3T4CNHPGOV9/fQ2PPvo2779/BSUl6Z0soAIzC5imSU1NTZMCU0rJDTe8yYYNER5+eEiT+/5lJAK7dsL2LbDpR/RtP6Lt3g6xajQnBtJMhKLjHnC6k9jKLKUFqaV0saa2GpPLMJKtSkNLdMcKMHTQDQjkIfPaIkMHIfO6IPM6g9HBvelFTfqZFMV3HAvd3EYg8gN51V8SqvwXgZp1GPEKtHgE4TjIRCi6N+mucLFx9/6zAKkjHc0NVUe6+++KPJxAKfGiQ4h2OZ7owcdjlh2GU1C3C3fv7to9M/dbNzzvuec1AgGbP/xhUFof12fXwgQVmD/VnItI33ffByxZsoHf/vZiCgoOPFFFOg7O7kqc8h04m7fAxvWIzVvQK3ehW1EMOw7YaHqiVVi7v2diHaJu79nv08BtXSa6Tt3AFO7HvUJTg4AOgQAyVASF7ZElXXAKD4aCgyDUAQJtQFNjiopSl7DCGJENhHZ/R/7uTwnu/hqjegsiHkaYltvSdBJrm21RuwuVtMSevfEdA+m4rVBNChyCWPllWO26Ez7kZ0QPORarrCsyL7mUxdtlKeGwyahRzzF79qlcfnn69r5WgZkFmhqYf/zjVzz44Cf88Y8jKCv76Ww5x7SwK3Zhbt+B9eNG7I0b+f/t3WtsXOWdx/HvuczNY8fO1YmTNLSUhFsJEErFpipotaxUusutFZBKdFm1KlVZVayWSpsKZV+AULvdF121okiUywJiV2yDSlYqhFYroRa6zRZICAEaFkIw4Bg3ji+xPZdznmdfHM94Zjx2xveZ8e8jWbHHc47HoMwvz+X//E1vL+7wMG42i5fP4xmD71o8Lxrsua7FccEphp4DbgCeF61L+m4UiK6FmBsVcMeI/rfGx2+STGLTK7Ar1kJ7F6ZjI6TXQXoNxDVaFJkTE+Jm+kgMvUPi5Gsk+w4TH/4ANzOEk8uCcbChnZjONWZ8sxDRqNTxsEH0tWPAGhfrpQnSq8luOJexT1xCdvP5hCvXgx9jqaZrjxzp5Y47nuGll25iy5YV83LPwolqDdI8GhSYk82mifRzzx3n299+gZ/97Aa2bOkgyGTJ9feT6/uYbHc32Q8/wpz8E4yM4mWzePkcjrV4joNnbLScGBp818HzLD4Wz3dwvegcHGf8jE4KfxZGmR4Q86OPVAKbbsOu6oQ1XZg1m6B1DbSthvhCnKEpIpNYi5sdIDb4Hom+N0ieOEj81DHcsQG8XA4bhmCiNU/HFtY+idqlhYCJuufYMHr/cfAJYy3k2zeQOetiMp+6hNyGT2HaOiaOIFmkspSHH/4Dv//9MX71qxvwvLmXmjRY82hQYE5mjGFoaKjmwPyf37zHzV/9b/7p77axJTFI5oMPyPWfwhkdhVwWLwyLM6JeGOI6Dn5oohlVwDcG33HHZ1Ytvge+Y/E8Z2KE6RGFZTwGyQS2rQ27eh10dmHWdcGqTlixChIKRpF642SHiQ0eJ9H7BokPD5H807v4wychn41CEne87tOOH+fojH8A1h3vP+pC4GC8OKalg+z6s8mccwmZsy8iv6oTEskFL0sJQ8M3v/k0X/ziZnbvnvvufwVmE7DWMjg4OG1gmiCg99Ahfvvkfv7h8RXcdOEQl6w6DWGIa6MegFG1hsU1Bg8Hzxr88SmI2Hhguq4bBarjjO/HsfieixfzcVvieC0p3LWrYcNGTNdmWLsOOlZBuhXqoEmziMycmxnGP/U+iZ4jpN5/lVjfMfzTg5AZw+LgGBdConNyx6dzKUzhWheDG+2+xSOMp8iuXE/m7AvIbt1BsOmT0L4KFmi69sSJYW655d/5xS/+is9+dv2c7qXAbAKFwCwcRF7NiYMHeeXnz3LnI61cdXaOnWv6xoMyCkjXccbLHG20KdWZmEF1HKLgxIn258Tj+MkkfkcH/vr1+Js24a5fj7NmDW5HR7RrVUSaljM6QPzk+8TfP0zLsVfwT36APzwIYTAx0jQOhAYMOMbFhpYQD+s4GOsSBA7Gi5FrW01+4yfJn38p4dnnQecm3HhhY8P82L//be6//0UOHLiF1tbZrz8aY4jH4/N2DOkiUGBWMzAwMGVgWmM4vPe/+Pqej/j0Wpdrtw6R6z8J1uKEIb7jRGuTTGxMdSx4vkcsHsdPpUisWUNy40ZiGzcR61yHv2oVbksLS326h4gsMWvxhk8SP/F/xN85ROr4a8RP9kAmA7n8xAjURM0UrHEIjTNeEuoS2qiFt8ElbG0jWLuB4PwdsO0inE98CieZYj7eZ/bs+RWtrZYHH/yLWd9DgdkkBgcHS9YCyuXzIdf/5ZM4ecPf7AwIBgbInjoFuVw0mnTdaJohkSTRmqals5OWTZtIbegitnYt8Y4OnMaZghCRpZTP4w30knjvDVLvvEqi+228/j7I56PuQ9YhDCA0htA4hNbBWAfjgnF8QiwhDkFLC6ZjNfacz8D5l+KfdQ7OipWzPlhldDTHTTc9yX33XcGXv3zOrO6hwGwSQ0NDQFTzdODAAS699FJc18UYw2OPvcWdd77I5s4YCSePZ3L4hCQTPi1tKdIdbaTb22hpT5NqS5NMxUgk/PEPr/h5MukRj/skkxPfSyb94mOx2NRTwiKyPLmjw3gfd5N4+yDJPx7C7f0Ae3oYkzPRMqeNGg9FG29ttAnX8wkdB+M40T6ieBzTthLO2obzmcuIfXIb7qq1ODMMrsOHT/Cd7+zjd7+7mc2bZ94/r8GaR4MCs7rh4eFix4/77ruPZ599lttvv52bb74Za126u0c53T/EQG8/o8Nj2NQKjJckm7NkMgFjYwFjYyGZTEAmEzA6Go4/FpDNhsXvj41F3y88Fj0/ui4M7XjAesVQLQ3Xia+j4K0M5MowLg3qiQD3y+4fj3sKaZFGEYa4p/rw3/sj/psv477zR5yBfmxmLJqSdaPlzzA00VKo548fNuRgPJfQcaMAbV+Fs+ksYhdeRuzTF+Ct3YBbY3g++OABXn31fZ577voZl5oYY0gkEgrMRjc0NEQ+ny8eRzUwMMCPf/xjfv3rX/ODH/yAL3zhCwv+GsLQkMmUB2n0+eTwjUI6LHve6GhQfH7h+6Vfl16TzUZ/5nIh8XgUoNFH+Si4ELZRQJeGs0ciESsL38pgL9x3IrgL1/klNWQiMmujI7gfHcd76xDOW69iPz6BOT0YhabrYnEwNoz2EBkwnoMZH32GocH4PqTbcLs+Qfy8S0iddwmxtRumXUIKQ8M3vrGX6647i+9+d8eMXq4Cs0kEQUAmkylrl+U4Dn19fWQyGTZv3rzUL3FBGGPLRrmlo+DS8K4M52gUXRrmYcVoeuK6bLb0/hMhXTraLYycq01lTwT05OAtHYlXPlY+ylZIS5MLAvj4Q3j3KPaNP2C6j2EH+zFYrOtighDjWAxRCaj1fIw1GM8jdBys40K6FW9dF4kLLqNl62eIdW7Cq3IqT0/PELt2/Qf79v01O3Z01vwSG6x5NCgwp2atJQxDcrkcYRhWFAXLfLHWloyUw4pR8eRRcuHxbDYsjqIrR8+Fx7LZyusnRtOxmDsppKPRdPUwLjx3Yvp76jCuDPjCSH2hGvGKTMtazMBJzAfH4PAfMMfeIuzvIwiyGBxCa7COF617WkNoLcZ1sbE4xpro2L50G+76jaS2XkTb+TtIrN+IG5sIz+eeO8oDD/yOAwduIZ2ubVOjArMJKTibj7WWXM5MCtNqo+fKEXZpOFeOnitH3aXT35lMiOs6xUAuDdZqwVv4vDSgq61fVwZz5XNjsYbZgSiLxIycxvS8T/DWQfJvHiLff4JwZBTrudHaZj6PBYy1mFgCa8LozAQvhnEdnESKWGcXredeyorzLyaxtgs3Hufuu/ezcqXHAw/8+RlfQ6GnrwKzSSk4ZS6steTz5evSU42eK0fKpdPj1TaPlU+jT3yezYbjb0rlQTv9lHf1oC5dvy4P/fL16WQyGknr70VjMLksYU83maOvE7z5CrmPPyIYG8FiCY3B4GJ9jzCXI8RGfXFjSYzJYxwXJ9FCvHMjbRfswNu0la/f+Rv++V+u5Lrrzp725zZgL0xQYM6cglMaSRCYKUN28gh78tR26dp0+ai6fERdujZd2OFdLZxLw7Z8utsnFqs+JT7VRrLyqXLt8J4rGwSEfzpB5u3XGTnyMrmeboKR4Wiq1nMJgwATBlgvBp6PyWcgkcQ4LqEBt7WVU3YFDz8f8NRv/56ztm2Y+mcpMJcXBadIdWFoysqqzrQJrHJzWena9OTp8YlrJqa7A/J5UxHI0013Tw7r6mvRkx8rD/DmDWkThgQnexl79y1Ov/4y2Z7j5E8PRtO0QRj108USZLOErofb0ooxeWJtqzh43PDOUBv/9sI/kmhNV71/A/bCBAXm3FULzgbaJi3SFAo7vKeb2q4M5/LNY5UbzyZfVwjnQvCX7vBOJr1JIVs6Ci7dCT65Trp6GE+1kWyxd3jbMCTf38fIu29x+sjLjH70HsHwICHR5qAwl8V6Hn7bSmJt7STWdvHQz9/nz758JXfdc031eyowlzcFp8jyYszEDu/K0fOZ6qZHRyevTVdfzw7KdpFns9EO7/LDTKYaBZcfahKPx6quX09dOz0xUi8cSmCtJd/fx+l3jzLw+gFGPzxOfnSYMAjw29tJdW7GS7cyFl/L7n89xn/u/1suvnjtpP92Ddg8GhSY80/BKSILpXSH95mmtit3cFduHJuq3rra2rTnOWVhnEz6JOIuK7wRVgU9rMwcJ2UGsS0rMSs24LSu4X97VpEJLK+88tVJu7UbsLUXTBGY6ic1B47j4Ps+nucRBIGCU0TmTTSNGR2Z2dGxONOZhR3elfXNZWvMI1mGe3sZ7T9F3vjkEivZEng4DuTzeVwXuru7eeqpp9i5cydXXHHForz2xVA3gbl79246Ozu58847l/qllLn88st55JFHuOCCC6Z8juM4xGIxfN9XcIpIw3Ich3g82onc3j7dMyeXkvT29vLwwz9j7969pFIpvvKVr3Duuecu2GtdCnXxbt7X18djjz3G7bffDsBPfvITLrvsMhKJBLfddlvN9+nv7+eGG24gnU6zZcsWnnzyyTlfd9ddd7Fnz56a7lMIzpaWlmKRbhiGxcPdRUSa1QMPPIC1lieeeIJ9+/Zx66230tbWVjyruxnUxRrmD3/4Q44ePcqDDz4IwNNPP43ruuzfv5+xsTEeffTRmu6za9cujDE89NBDHDx4kC996Uu89NJL044Oz3RdJpOhq6uLI0eOsGHD1LVG1VhriyNOY8yUjapFRJpBIU8Kp/t4nofv+/i+32izbVXfqOviN3j22We58sori1/feOONXH/99axevbrme4yMjLB3717uueceWltb+fznP8+1117L448/PqfrkskkO3bs4Pnnn5/x71U54oRoAfwM/0gREWkohQ2Qhdm0eDxOOp2mpaWl0fpgTqsufovDhw+zbdu2Od3j6NGjeJ7H1q1bi49t376dI0eOzPm68847j0OHDs36tZUGZ6EWScEpIo2sWki2tLQ0XUiWqotNPwMDA7S1zbyLd6nTp0/TXrFK3d7ezvDw8Jyva2tro6enZ06vD6beHKSpWhFpBIWp1sJhBPF4vDjduhzew+oiMFeuXHnGYDuT1tZWhoaGyh4bGho6YxDXct3w8DAdHR1zen2lFJwi0igKAVlQWJP0vOY9LnAqdTFmvuiiizh69Oic7rF161aCIODtt98uPnbo0KEzbvip5bo333yT7du3z+n1VVNtjTMMQ03VisiSstZijClOuXqeRzKZJJ1Ok0wm8X1/2YUl1ElgXnPNNbzwwgvFr4MgIJPJEIYhYRiSyWQIgqD4/dtuu21SuUk6nebGG29kz549jIyM8OKLL/LMM89w6623zum6bDbLyy+/zNVXXz3/v/g4BaeILLXKkHRdl2QySWtrK6lUatmGZKm6CMyvfe1r/PKXv2RsbAyAe++9l1Qqxfe//32eeOIJUqkU9957b/H53d3d7Ny5c9J97r//fsbGxli3bh27du3ipz/9adlIcTbX7du3j6uuuoqurq75/rUnUXCKyGIqTLcaY4ohmUgkijtcY7HYsg/JUnVRhwnwve99j3Xr1p3xpJ9cLsf27dt57bXXZnQ24Wyv+9znPsdDDz3EhRdeWPM18yU6pipPPp9XHaeIzJtCSAK4rlvcU9GMO1tnSYevNyoFp4jMVSEkC718Sw8U0PvJJArMRqfgFJGZqCwDKR1J6r1jWvV70o/UplD3VHoAgtY4RaRU6ZqktRbf90mlUqTTaRKJxKKUg+zevZsf/ehHC/ozZuPyyy8/42E201FgNiAFp4iUqqcykMpmGrNtirEQTThm0kyjmro4uEBmpxCcsVisOFWrAxBElofS6VYAz/OKU65L+ff/0Ucf5ZprriGVSgFwxx13EI/H6e3tLTa32L59+xlr5Lu6urj77ruLTThqNd3Pu/baa/nWt75FT0/PjJtpgEaYTUEjTpHloRHKQEqbacy2KQYsTBOOuTTTAAVmU6kMzsLhyApOkcbWSN1ASptpzLYpxmwtdDMNTck2ocqpWvXjFGk8lWUgjXLQeWkzjdk2xZithW6mocBsYgpOkcbSDN1ASptpzLYpxmwtdDON+hnHy4Ip/MUrbCvXVK1I/aiHMpD5VNpMY7ZNMWZroZtpKDCXEQWnSH2opzKQ+VbaTGO2TTFgYZpwzLWZhgJzGVJwiiy+5dINpLKZxmybYixEE465NtPQ0XhSPHIvl8thrW2o9RKRelZ4fy2sS3qeV3aGa7NqgmYaOktWplcanEBxd56IzIy6gTQ8BabURsEpMnPqBtJUFJgyMwpOkempG0jTUmDK7Cg4RSaUnt8K4Ps+sVhMIdlcFJgyNwpOWa4qDzovhGSj1UhKzRSYMj+steRyOfL5PKDglOZUr91AZFEoMGV+KTil2SzXMhCZRIEpC0PBKY1OZSBSQYEpC0vBKY1EZSAyDQWmLA5jDPl8XsEpdUdlIFIjBaYsLgWn1AOVgcgsKDBlaZQGp86qlcWgMhCZIwWmLK1CcBbqOBWcMp9UBiLzSIEp9UHBKfNFZSCyQBSYUl8UnDJbKgORBabAlPqk4JRaqAxEFpECU+qbglMqqQxElogCUxqDgnN5UxmI1AEFpjQWBefyoTIQqTMKTGlMCs7mpDIQqWMKTGlsxpiys2oVnI1HZSDSIBSY0hwKwRkEgU4OahAqA5EGo8CU5qLgrG8qA5EGpsCU5qTgrB8qA5EmocCU5qbgXBoqA5EmpMCU5UHBufBUBiJNToEpy4uCc36pDESWEQWmLE+lwQlqZD0TKgORZUqBKcubgrN2KgORZU6BKQLlByAUQlPBqTIQkRIKTJFSCk6VgYhMQYEpUs1yC06VgYickQJTZDphGBbXOJstOFUGIjIjCkyRWjRLcKoMRGTWFJgiM9GIwakyEJF5ocAUmY1GCE6VgYjMKwWmyFzUW3CqDERkwSgwRebDUganykBEFoUCU2Q+LVZwqgxEZNEpMEUWwkIEp8pARJaUAlNkIc01OFUGIlI3FJgiC62wEafW4FQZiEhdUmCKLJZCcGazWcIwnBScKgMRqWsKTJHFVhmcgMpAROqfAlNkqRSCMwgChaRI/VNgioiI1KBqYGrBREREpAYKTBERkRooMEVERGqgwBQREamBAlNERKQGCkwREZEaKDBFRERqoMAUERGpgQJTRESkBgpMERGRGigwRUREaqDAFBERqYECU0REpAYKTBERkRooMEVERGqgwBQREamBf4bvqyW8iIgIGmGKiIjURIEpIiJSAwWmiIhIDRSYIiIiNVBgioiI1ECBKSIiUoP/BwkzJbhSvKVIAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "filenames": { + "image/png": "/home/john/gh_synced/quantecon/continuous_time_mcs/ctmc_lectures/_build/jupyter_execute/ergodicity_5_0.png" + }, + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "def unit_simplex(angle):\n", + " \n", + " fig = plt.figure(figsize=(8, 6))\n", + " ax = fig.add_subplot(111, projection='3d')\n", + "\n", + " vtx = [[0, 0, 1],\n", + " [0, 1, 0], \n", + " [1, 0, 0]]\n", + " \n", + " tri = Poly3DCollection([vtx], color='darkblue', alpha=0.3)\n", + " tri.set_facecolor([0.5, 0.5, 1])\n", + " ax.add_collection3d(tri)\n", + "\n", + " ax.set(xlim=(0, 1), ylim=(0, 1), zlim=(0, 1), \n", + " xticks=(1,), yticks=(1,), zticks=(1,))\n", + "\n", + " ax.set_xticklabels(['$(1, 0, 0)$'], fontsize=12)\n", + " ax.set_yticklabels(['$(0, 1, 0)$'], fontsize=12)\n", + " ax.set_zticklabels(['$(0, 0, 1)$'], fontsize=12)\n", + "\n", + " ax.xaxis.majorTicks[0].set_pad(15)\n", + " ax.yaxis.majorTicks[0].set_pad(15)\n", + " ax.zaxis.majorTicks[0].set_pad(35)\n", + "\n", + " ax.view_init(30, angle)\n", + "\n", + " # Move axis to origin\n", + " ax.xaxis._axinfo['juggled'] = (0, 0, 0)\n", + " ax.yaxis._axinfo['juggled'] = (1, 1, 1)\n", + " ax.zaxis._axinfo['juggled'] = (2, 2, 0)\n", + " \n", + " ax.grid(False)\n", + " \n", + " return ax\n", + "\n", + "Q = np.array(Q)\n", + "ψ_00 = np.array((0.01, 0.01, 0.99))\n", + "ψ_01 = np.array((0.01, 0.99, 0.01))\n", + "ψ_02 = np.array((0.99, 0.01, 0.01))\n", + "\n", + "ax = unit_simplex(angle=50) \n", + "\n", + "def flow_plot(ψ, h=0.001, n=300, angle=50):\n", + " colors = cm.jet_r(np.linspace(0.0, 1, n))\n", + "\n", + " x_vals, y_vals, z_vals = [], [], []\n", + " for t in range(n):\n", + " x_vals.append(ψ[0])\n", + " y_vals.append(ψ[1])\n", + " z_vals.append(ψ[2])\n", + " ψ = ψ @ expm(h * Q)\n", + "\n", + " ax.scatter(x_vals, y_vals, z_vals, c=colors, s=20, alpha=0.2, depthshade=False)\n", + "\n", + "flow_plot(ψ_00)\n", + "flow_plot(ψ_01)\n", + "flow_plot(ψ_02)\n", + "\n", + "# Add stationary distribution\n", + "P_1 = expm(Q)\n", + "mc = qe.MarkovChain(P_1)\n", + "ψ = mc.stationary_distributions[0]\n", + "ax.scatter(ψ[0], ψ[1], ψ[2], c='k', s=30, depthshade=False)\n", + "\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This black dot is the stationary distribution $\\psi^*$ of the Markov semigroup $(P_t)$ generated by $Q$.\n", + "\n", + "It was calculated the ``stationary_distributions`` attribute of QuantEcon's\n", + "[``MarkovChain`` class](https://quanteconpy.readthedocs.io/en/latest/markov.html), by arbitrarily setting $t=1$ and solving $\\psi P_1 = \\psi$.\n", + "\n", + "Below we show that, for this choice of $Q$, the stationary distribution\n", + "$\\psi^*$ is unique in $\\dD$, due to irreducibility.\n", + "\n", + "Moreover, $\\psi P_t \\to \\psi^*$ as $t \\to \\infty$ for any $\\psi \\in \\dD$, as\n", + "suggested by the figure.\n", + "\n", + "\n", "\n", - "The next result makes this possible.\n", + "### Stationarity via the Generator\n", "\n", - "It is analogous to the idea that a point $\\bar x$ in $\\RR^d$ is stationary for\n", + "In many cases, it is easier to use the generator of the semigroup to identify\n", + "stationary distributions rather than the semigroup itself.\n", + "\n", + "This is analogous to the idea that a point $\\bar x$ in $\\RR^d$ is stationary for\n", "a vector ODE $x'_t = F(x_t)$ when $F(\\bar x) = 0$.\n", "\n", - "It holds true under weaker conditions but the version stated here is easy to\n", - "prove and sufficient for most cases we consider.\n", + "(Here $F$ is the infinitesimal description, and hence analogous to the generator.)\n", + "\n", + "The next result holds true under weaker conditions but the version stated here\n", + "is easy to prove and sufficient for applications we consider.\n", + "\n", "\n", "```{proof:theorem}\n", ":label: statfromq\n", "\n", - "For a conservative intensity matrix $Q$ on $S$ and \n", - "associated Markov semigroup $(P_t)$, a distribution $\\psi$ on $S$ is stationary \n", - "for $(P_t)$ if and only if $\\psi Q = 0$.\n", + "Let $(P_t)$ be a UC Markov semigroup with generator $Q$. A distribution\n", + "$\\psi$ on $S$ is stationary for $(P_t)$ if and only if $\\psi Q = 0$.\n", "```\n", "\n", "```{proof:proof}\n", - "Suppose first that $\\psi \\in \\dD$ and $\\psi Q = 0$.\n", + "Fix $\\psi \\in \\dD$ and suppose first that $\\psi Q = 0$.\n", "\n", - "Since $Q$ is conservative, $(P_t)$ is a UC semigroup with $P_t = e^{tQ}$ for\n", - "all $t$, and hence, for any $t \\geq 0$,\n", + "Since $(P_t)$ is a UC Markov semigroup, we have $P_t = e^{tQ}$ for all $t$,\n", + "and hence, for any $t \\geq 0$,\n", "\n", "$$\n", " \\psi e^{tQ} = \\psi + t \\psi Q + t^2 \\frac{\\psi Q^2}{2!} \n", @@ -97,11 +262,11 @@ "$$\n", "\n", "Since $(P_t)$ is UC and $Q$ is its generator, we have $\\| D_t - Q \\| \\to 0$ in\n", - "$\\lL(\\ell_1)$ as $t \\to 0$ from the right.\n", + "$\\lL(\\ell_1)$ as $t \\to 0$.\n", "\n", "Hence $\\| \\psi Q \\| \\leq \\liminf_{t \\to 0} \\| \\psi D_t \\|$.\n", "\n", - "As $\\psi$ is stationary, $\\psi D_t = 0$ for all $t$.\n", + "As $\\psi$ is stationary for $(P_t)$, we have $\\psi D_t = 0$ for all $t$.\n", "\n", "Hence $\\psi Q = 0$, as was to be shown.\n", "```\n", @@ -115,36 +280,45 @@ "We say that state $y$ is **accessible** from state $x$ if\n", "there exists a $t \\geq 0$ such that $P_t(x, y) > 0$.\n", "\n", - "A Markov semigroup $(P_t)$ on $S$ is called **irreducible** if $y$ is\n", - "accessible from $x$ for every $(x,y) \\in S \\times S$.\n", + "We say that $x$ and $y$ **communicate** if $x$ is accessible from $y$ and $y$\n", + "is accessible from $x$.\n", + "\n", + "A Markov semigroup $(P_t)$ on $S$ is called **irreducible** if \n", + "every pair $x,y$ in $S$ communicates.\n", "\n", "We seek a characterization of irreducibility of $(P_t)$ in terms of its\n", "generator.\n", "\n", "As a first step, we will say there is a **$Q$-positive probability flow** from $x$\n", - "to $y$ if there exists a finite sequence $(z_i)_{i=1}^m$ in $S$ starting at\n", - "$x=z_1$ and ending at $y=z_m$ such that $Q(z_i, z_{i+1}) > 0$ for all $i$.\n", + "to $y$ if there exists a finite sequence $(z_i)_{i=0}^m$ in $S$ starting at\n", + "$x=z_0$ and ending at $y=z_m$ such that $Q(z_i, z_{i+1}) > 0$ for all $i$.\n", "\n", "\n", - "```{proof:lemma}\n", + "```{proof:theorem}\n", ":label: equivirr\n", "\n", + "Let $(P_t)$ be a UC Markov semigroup with generator $Q$.\n", "For distinct states $x$ and $y$, the following statements are equivalent:\n", "\n", "1. The state $y$ is accessible from $x$ under $(P_t)$.\n", - "1. There is a positive probability flow from $x$ to $y$ under $Q$.\n", + "1. There is a $Q$-positive probability flow from $x$ to $y$.\n", + "1. $P_t(x, y) > 0$ for all $t > 0$.\n", "```\n", "\n", - "((prove only for the UC case?))\n", "\n", "```{proof:proof}\n", - "For distinct states $x$ and $y$, we have\n", + "Pick any two distinct states $x$ and $y$.\n", + "\n", + "It is obvious that statement 3 implies statement 1, so we need only prove \n", + "(1 $\\implies$ 2) and (2 $\\implies$ 3).\n", + "\n", + "Starting with (1 $\\implies$ 2), recall that\n", "\n", "$$\n", " P_t(x, y) = t Q(x,y) + \\frac{t^2}{2!} Q^2(x, y) + \\cdots\n", "$$ (ptexpan)\n", "\n", - "If $x$ is accessible from $y$, then $P_t(x, y) > 0$ for some $t > 0$, and hence\n", + "If $x$ is accessible from $y$, then $P_t(x, y) > 0$ for some $t > 0$, so\n", "$Q^k(x,y) > 0$ for at least one $k \\in \\NN$.\n", "\n", "Writing out the matrix product as a sum, we now have\n", @@ -159,54 +333,406 @@ " > 0\n", "$$ (qkassum)\n", "\n", - "It follows that at least one element of the sum must be strictly positive, so\n", - "positive probability flow from $x$ to $y$ is established.\n", + "It follows that at least one element of the sum must be strictly positive.\n", "\n", - "To prove the converse, suppose there is positive probability flow from $x$ to\n", - "$y$.\n", + "Therefore, a $Q$-positive probability flow from $x$ to $y$ exists.\n", "\n", - "We can then take a finite sequence $(z_i)_{i=1}^m$ starting at $x$ and ending\n", - "at $y$ with $Q(z_i, z_{i+1}) > 0$ for all $i$.\n", + "Turning to (2 $\\implies$ 3), first note that, for arbitrary states $u$ and $v$, if $Q(u, v) > 0$ then $P_t(u, v) > 0$ for all $t > 0$.\n", "\n", - "We can and do assume that $(z_i)_{i=0}^m$ is the shortest such sequence\n", - "(otherwise just pick the shortest one).\n", + "To see this, let $(\\lambda, K)$ be the jump chain pair constructed from $Q$ via\n", + "{eq}`lambdafromq`, {eq}`kfromqxx` and {eq}`kfromqxy`.\n", "\n", - "Because any shorter sequence $(u_i)_{i=1}^k$ linking $x$ and $y$ has $Q(u_i, u_{i+1}) = 0$ for some $i$, {eq}`qkassum` implies that $Q^k(x,y) = 0$ for all $k < m$.\n", + "Observe that, since $Q(u, v) > 0$, we must have $\\lambda(u) > 0$.\n", "\n", - "Hence, by {eq}`ptexpan`, we have $P_t(x, y) = t^m Q^m(x, y) + o(t^m)$.\n", + "As a consequence, applying {eq}`kfromqxy`, we have \n", "\n", - "As $Q^m(x, y) > 0$, we see that $P_t(x, y) > 0$ for sufficiently small $t$.\n", + "$$\n", + " K(u, v) = \\frac{Q(u, v)}{\\lambda(u)} > 0\n", + "$$\n", "\n", - "```\n", "\n", "\n", - "Irreducible implies aperiodic.\n", + "Let $(Y_k)$ and $(J_k)$ be the embedded jump chain and jump sequence generated\n", + "by {proof:ref}`ejc_algo`, with $Y_0 = u$.\n", "\n", - "```{proof:theorem}\n", - "For a ((UC)) Markov semigroup $(P_t)$, the following statements are\n", + "With $E_1 \\sim \\Exp(1)$ and $E_2 \\sim \\Exp(1)$, we have, for any $t > 0$,\n", + "\n", + "$$\n", + "\\begin{aligned}\n", + " P_t(u, v) \n", + " & \\geq \\PP \\{J_1 \\leq t, \\, Y_1 = v, \\, J_2 > t \\}\n", + " \\\\\n", + " & \\geq \\PP \\{E_1 \\leq t\\lambda(u), \\, E_2 > t\\lambda(v) \\} \\PP\\{ Y_1 = v \\}\n", + " \\\\\n", + " & = \\PP \\{E_1 \\leq t\\lambda(u)\\} \n", + " \\PP \\{ E_2 > t\\lambda(v) \\} K(u, v) \n", + " \\\\\n", + " & > 0\n", + "\\end{aligned}\n", + "$$\n", + "\n", + "Now suppose there is a $Q$-positive probability flow $(z_i)_{i=0}^m$ from $x$\n", + "to $y$.\n", + "\n", + "If we fix $t > 0$ and repeatedly apply {eq}`chapkol_ct2` along with the last\n", + "result, we obtain\n", + "\n", + "$$\n", + " P_t(x, y)\n", + " \\geq\n", + " \\prod_{i=0}^{m-1} P_{t/m} (z_i, z_{i+1}) > 0\n", + "$$\n", + "\n", + "\n", + "\n", + "```\n", + "\n", + "{proof:ref}`equivirr` leads directly to the following strong result.\n", + "\n", + "```{proof:corollary}\n", + ":label: perimposs\n", + "For a UC Markov semigroup $(P_t)$, the following statements are\n", "equivalent:\n", "\n", "1. $(P_t)$ is irreducible.\n", "1. $P_t(x,y) > 0$ for all $t > 0$ and all $(x,y) \\in S \\times S$.\n", "```\n", "\n", + "\n", + "```{note}\n", + "To obtain stable long run behavior in discrete time Markov chains, it is\n", + "common to assume that the chain is aperiodic.\n", + "\n", + "This needs to be assumed on top of irreducibility if one wishes to rule out\n", + "all dependence on initial conditions.\n", + "\n", + "{proof:ref}`perimposs` shows that periodicity is not a concern for irreducible\n", + "continuous time Markov chains.\n", + "\n", + "Positive probability flow from $x$ to $y$ at some $t > 0$ immediately implies\n", + "positive flow for all $t > 0$.\n", + "\n", + "```\n", + "\n", + "\n", + "\n", + "\n", + "## Asymptotic Stabiltiy\n", + "\n", + "We call Markov semigroup $(P_t)$ **asymptotically stable** if $(P_t)$ has a\n", + "unique stationary distribution $\\psi^*$ in $\\dD$ and\n", + "\n", + "$$\n", + " \\| \\psi P_t - \\psi^* \\| \\to 0 \\text{ as } t \\to \\infty\n", + " \\text{ for all } \\psi \\in \\dD\n", + "$$ (asyms)\n", + "\n", + "Our aim is to establish conditions for asymptotic stability of Markov\n", + "semigroups.\n", + "\n", + "\n", + "\n", + "### Contractivity\n", + "\n", + "Let's recall some useful facts about the discrete time case.\n", + "\n", + "First, if $P$ is any Markov matrix, we have, in the $\\ell_1$ norm,\n", + "\n", + "$$\n", + " \\| \\psi P \\| \\leq \\| \\psi \\|\n", + " \\text{ for all } \\psi \\in \\dD\n", + "$$ (allmocontract)\n", + "\n", + "This is because, given $\\psi \\in \\dD$, \n", + "\n", + "$$\n", + " \\| \\psi P \\|\n", + " = \\sum_y \\left| \\sum_x \\psi(x) P(x, y) \\right|\n", + " \\leq \\sum_y \\sum_x | \\psi(x) | P(x, y)\n", + " = \\| \\psi \\|\n", + "$$\n", + "\n", + "By linearity, for $\\psi, \\phi \\in \\dD$, we then have\n", + "\n", + "$$\n", + " \\| \\psi P - \\phi P \\| \\leq \\| \\psi - \\phi \\|\n", + "$$ \n", + "\n", + "Hence every Markov operator is contracting on $\\dD$.\n", + "\n", + "Moreover, if $P$ is everywhere positive, then this inequality is strict:\n", + "\n", + "```{proof:lemma} Strict Contractivity\n", + ":label: strictcontract\n", + "\n", + "If $P$ is a Markov matrix and $P(x, y) > 0$ for all $x, y$, then \n", + "\n", + "$$\n", + " \\| \\psi P - \\phi P \\| < \\| \\psi - \\phi \\|\n", + " \\text{ for all } \\psi, \\phi \\in \\dD\n", + " \\text{ with } \\psi \\not= \\phi\n", + "$$ \n", + "\n", + "```\n", + "\n", + "The proof follows from the strict triangle inequality, as opposed to the weak\n", + "triangle inequality we used to obtain {eq}`allmocontract`.\n", + "\n", + "See, for example, Proposition 3.1.2 of {cite}`lasota1994chaos` or Lemma 8.2.3 of {cite}`stachurski2009economic`.\n", + "\n", + "### Uniqueness\n", + "\n", + "Irreducibility of a given Markov chain implies that there are no disjoint\n", + "absorbing sets.\n", + "\n", + "This in turn leads to uniqueness of stationary distributions:\n", + "\n", + "```{proof:theorem}\n", + ":label: uniirr\n", + "\n", + "Let $(P_t)$ be a UC Markov semigroup on $S$. If $(P_t)$ is irreducible, then\n", + "$(P_t)$ has at most one stationary distribution.\n", + "```\n", + "\n", + "\n", "```{proof:proof}\n", + "Suppose to the contrary that $\\psi$ and $\\phi$ are both stationary for\n", + "$(P_t)$.\n", + "\n", + "Since $(P_t)$ is irreducible, we know that $P_1(x,y) > 0$ for all $x,y \\in S$.\n", "\n", - "To be added.\n", + "If $\\psi \\not= \\phi$, then, due to positivity of $P_1$, the strict inequality\n", + "in {proof:ref}`strictcontract` holds.\n", + "\n", + "At the same time, by stationarity, $\\| \\psi P - \\phi P \\| = \\| \\psi - \\phi\n", + "\\|$. Contradiction.\n", "```\n", "\n", "\n", - "Irreducible implies uniqueness.\n", + "```{proof:example}\n", "\n", + "An M/M/1 queue with parameters $\\mu, \\lambda$ is a continuous time Markov chain $(X_t)$ on $S = \\ZZ_+$ with with intensity matrix\n", "\n", + "$$\n", + "Q=\\begin{pmatrix}\n", + " -\\lambda & \\lambda & 0 & 0 & \\cdots \\\\\n", + " \\mu & -(\\mu + \\lambda) & \\lambda & 0 & \\cdots \\\\\n", + " 0 & \\mu & -(\\mu + \\lambda) & \\lambda & \\cdots\\\\\n", + " \\vdots & \\vdots & \\vdots & \\vdots & \\ddots\n", + "\\end{pmatrix}\n", + "$$ (mm1q)\n", "\n", - "## Asymptotic Stabiltiy\n", + "The chain $(X_t)$ records the length of the queue at each moment in time.\n", + "\n", + "The intensity matrix captures the idea that customers flow into the queue at\n", + "rate $\\lambda$ and are served (and hence leave the queue) at rate $\\mu$. \n", + "\n", + "If $\\lambda$ and $\\mu$ are both positive, then there is a $Q$-positive\n", + "probability flow between any two states, in both directions, so the\n", + "corresponding semigroup $(P_t)$ is irreducible.\n", + "\n", + "{proof:ref}`uniirr` now tells us that $(P_t)$ has at most one stationary\n", + "distribution.\n", + "\n", + "```\n", + "\n", + "\n", + "### Stability from the Skeleton\n", + "\n", + "Recall the definition of asymptotic stability given in {eq}`asyms`.\n", + "\n", + "Analogously, we call an individual Markov operator $P$ asymptotically stable if \n", + "$P$ has a unique stationary distribution $\\psi^*$ in $\\dD$ and\n", + "$\\psi P^n \\to \\psi^*$ as $n \\to \\infty$ for all $\\psi \\in \\dD$.\n", + "\n", + "The next result gives a connection between discrete and continuous stability.\n", + "\n", + "The critical ingredient linking these two concepts is the contractivity in\n", + "{eq}`allmocontract`.\n", + "\n", + "```{proof:lemma}\n", + ":label: stabskel\n", + "\n", + "Let $(P_t)$ be a Markov semigroup. If there exists an $s > 0$ such that the\n", + "Markov matrix $P_s$ is asymptotically stable, then $(P_t)$ is asymptotically\n", + "stable with the same stationary distribution.\n", + "\n", + "```\n", + "\n", + "```{proof:proof}\n", + "\n", + "Let $(P_t)$ and $s$ be as in the statement of {proof:ref}`stabskel`.\n", + "\n", + "\n", + "Let $\\psi^*$ be the stationary distribution of $P_s$. Fix $\\psi \\in \\dD$ and \n", + "$\\epsilon > 0$.\n", + "\n", + "By stability of $P_s$, we can take an $n \\in \\NN$ such that\n", + "$\\| \\psi P_s^n - \\psi^* \\| < \\epsilon$.\n", + "\n", + "Pick any $t > sn$. Set $h := t - sn$. \n", + "\n", + "By the contractivity in {eq}`allmocontract` and $P_{sn} = P_s^n$, we have\n", + "\n", + "\n", + "$$\n", + " \\| \\psi P_t - \\psi^* \\|\n", + " = \\| \\psi P_{sn} P_h - \\psi^* P_h \\|\n", + " \\leq \\| \\psi P_{sn} - \\psi^* \\|\n", + " < \\epsilon.\n", + "$$\n", + "\n", + "Hence asymptotic stability holds\n", "\n", - "To be added.\n", + "```\n", + "\n", + "\n", + "### Stability via Drift\n", + "\n", + "In this section we address drift conditions, which are a powerful method for\n", + "obtaining asymptotic stability when the state space can be infinite. \n", + "\n", + "The idea is to show that the state tends to drift back to a finite set over\n", + "time.\n", + "\n", + "Such drift, when combined with the contractivity in\n", + "{proof:ref}`strictcontract`, is enough to give global stability.\n", + "\n", + "The next theorem gives a useful version of this class of results.\n", + "\n", + "```{proof:theorem}\n", + ":label: sdrift\n", + "\n", + "Let $(P_t)$ be a UC Markov semigroup with intensity matrix $Q$. If $(P_t)$ is\n", + "irreducible and there exists a function $v \\colon S \\to \\RR_+$, a finite set\n", + "$F \\subset S$ and positive constants $\\epsilon$ and $M$ such that\n", + "\n", + "$$\n", + " \\sum_y Q(x, y) v(y) \n", + " \\leq \n", + " \\begin{cases}\n", + " M & \\text{ if } x \\in F \\text{ and }\n", + " \\\\\n", + " - \\epsilon & \\text{ otherwise } \n", + " \\end{cases}\n", + "$$\n", "\n", - "Include Foguel alternative?\n", + "then $(P_t)$ is asymptotically stable.\n", + "```\n", + "\n", + "The proof of {proof:ref}`sdrift` can be found in {cite}`pichor2012stochastic`.\n", + "\n", + "```{proof:example}\n", + "\n", + "Consider again the M/M/1 queue on $\\ZZ_+$ with intensity matrix {eq}`mm1q`.\n", + "\n", + "Suppose that $\\lambda < \\mu$. \n", + "\n", + "It is intuitive that, in this case, the queue length will not \n", + "tend to infinity (since the service rate is higher than the arrival rate).\n", + "\n", + "This intuition can be confirmed via {proof:ref}`sdrift`, after setting $v(i) =\n", + "i$.\n", + "\n", + "Indeed, we have, for any $i \\geq 1$,\n", + "\n", + "$$\n", + " \\sum_{j \\geq 0} Q(i, j) v(j) \n", + " = (i-1) \\mu - i (\\mu + \\lambda) + (i+1) \\lambda\n", + " = \\lambda - \\mu\n", + "$$\n", + "\n", + "Setting $F=\\{0\\}$ and $\\epsilon = \\mu - \\lambda$, we see that the conditions\n", + "of {proof:ref}`sdrift` hold.\n", + "\n", + "Hence the associated semigroup $(P_t)$ is asymptotically stable.\n", + "\n", + "```\n", + "\n", + "\n", + "```{proof:corollary}\n", + ":label: sfinite\n", + "\n", + "If $(P_t)$ is an irreducible UC Markov semigroup and $S$ is finite, then\n", + "$(P_t)$ is asymptotically stable.\n", + "```\n", + "\n", + "A solved exercise below asks you to confirm this.\n", + "\n", + "## Exercises\n", + "\n", + "### Exercise 1\n", + "\n", + "Let $(P_t)$ be a Markov semigroup. True or false:\n", + "for this semigroup, every state $x$ is accessible from itself. \n", + "\n", + "\n", + "### Exercise 2\n", + "\n", + "Let $(\\lambda_k)$ be a bounded sequence in $(0, \\infty)$.\n", + "\n", + "A **pure birth process** starting at zero is a continuous time Markov process\n", + "$(X_t)$ on state space $\\ZZ_+$ with intensity matrix\n", + "\n", + "$$\n", + "Q=\\begin{pmatrix}\n", + " -\\lambda_0 & \\lambda_0 & 0 & 0 & \\cdots \\\\\n", + " 0 & -\\lambda_1 & \\lambda_1 & 0 & \\cdots \\\\\n", + " 0 & 0 & -\\lambda_2 & \\lambda_2 & \\cdots\\\\\n", + " \\vdots & \\vdots & \\vdots & \\vdots & \\ddots\n", + "\\end{pmatrix}\n", + "$$\n", + "\n", + "Show that $(P_t)$, the corresponding Markov semigroup, has no stationary\n", + "distribution.\n", + "\n", + "### Exercise 3\n", + "\n", + "Confirm that {proof:ref}`sdrift` implies {proof:ref}`sfinite`.\n", + "\n", + "\n", + "## Solutions\n", + "\n", + "### Solution to Exercise 1\n", + "\n", + "The statement is true. With $t=0$ we have $P_t(x,x) = I(x,x) = 1 > 0$.\n", + "\n", + "### Solution to Exercise 2\n", + "\n", + "Suppose to the contrary that $\\phi \\in \\dD$ and $\\phi Q = 0$.\n", + "\n", + "Then \n", + "\n", + "$$\n", + " (\\phi Q)(j) \n", + " = \\sum_{i \\geq 0} \\phi(i) Q(i, j)\n", + " = - \\lambda_j \\phi(j) + \\lambda_j \\phi(j+1) \n", + " = 0\n", + "$$\n", + "\n", + "It follows that $\\phi$ is constant on $\\ZZ_+$.\n", + "\n", + "But $\\dD$ contains no constant functions when the state space is infinite.\n", + "(Why?)\n", + "\n", + "Contradiction.\n", + "\n", + "### Solution to Exercise 3\n", + "\n", + "Let $(P_t)$ be an irreducible UC Markov semigroup and let $S$ be finite.\n", + "\n", + "Pick any positive constants $M, \\epsilon$ and set $v = M$ and $F = S$.\n", + "\n", + "We then have\n", + "\n", + "$$\n", + " \\sum_y Q(x, y) v(y)\n", + " = M \\sum_y Q(x, y) \n", + " = 0\n", + "$$\n", "\n", - "Thm 6.2 of RR and MTK." + "Hence the drift condition in {proof:ref}`sdrift` holds and $(P_t)$ is\n", + "asymptotically stable." ] } ], @@ -239,8 +765,12 @@ }, "source_map": [ 13, - 30, - 40 + 46, + 58, + 86, + 90, + 98, + 166 ] }, "nbformat": 4, diff --git a/_sources/ergodicity.md b/_sources/ergodicity.md index 3ce94c5..aed6578 100644 --- a/_sources/ergodicity.md +++ b/_sources/ergodicity.md @@ -18,12 +18,28 @@ kernelspec: ## Overview -To be added. +In this lecture we discuss stability and equilibrium behavior for continuous +time Markov chains. + +To give one example of why this theory matters, consider queues, which are +often modeled as continuous time Markov chains. + +Queueing theory is used in applications such as + +* treatment of patients arriving at a hospital +* optimal design of manufacturing processes +* requests to a file server +* air traffic +* customers waiting on a help line + +A key topic in queueing theory is average behavior over the long run. + +* Will the length of the queue grow without bounds? +* If not, is there some kind of long run equilibrium? +* If so, what is the average waiting time in this equilibrium? +* What is the average length of the queue over a week, or a month? -Use the distribution flow figures from markov_prop.md, this time starting from -many different distributions. -Suggests ergodicity. We will use the following imports @@ -33,7 +49,9 @@ import scipy as sp import matplotlib.pyplot as plt import quantecon as qe from numba import njit +from scipy.linalg import expm +from matplotlib import cm from mpl_toolkits.mplot3d import Axes3D from mpl_toolkits.mplot3d.art3d import Poly3DCollection @@ -43,39 +61,149 @@ from mpl_toolkits.mplot3d.art3d import Poly3DCollection ## Stationary Distributions -Let $(P_t)$ be a Markov semigroup on countable state space $S$. +### Definition + +Let $S$ be countable. + +Recall that, for a discrete time Markov chain with Markov matrix $P$ on $S$, a +distribution $\psi$ is called stationary for $P$ if $\psi P = \psi$. -A distribution $\psi \in \dD$ is called **stationary** for $(P_t)$ if +This means that if $X_t$ has distribution $\psi$, then so does $X_{t+1}$. + +For continuous time Markov chains, the definition is analogous. + +Given a Markov semigroup on $S$, a distribution $\psi^* \in \dD$ is called +**stationary** for $(P_t)$ if $$ - \psi P_t = \psi + \psi^* P_t = \psi^* \text{ for all } t \geq 0 $$ -In many cases, it is easier to use the generator of the semigroup to identify -stationary distributions. +As one example, we look {ref}`again ` at the chain on $S = \{0, 1, +2\}$ with intensity matrix + +```{code-cell} ipython3 +Q = ((-3, 2, 1), + (3, -5, 2), + (4, 6, -10)) +``` + +The following figure was shown before, except that now there is a black dot +that the three trajectories seem to be converging to. + +(Recall that, in the color scheme, trajectories cool as time evolves.) + + +```{code-cell} ipython3 +:tags: [hide-input] + +def unit_simplex(angle): + + fig = plt.figure(figsize=(8, 6)) + ax = fig.add_subplot(111, projection='3d') + + vtx = [[0, 0, 1], + [0, 1, 0], + [1, 0, 0]] + + tri = Poly3DCollection([vtx], color='darkblue', alpha=0.3) + tri.set_facecolor([0.5, 0.5, 1]) + ax.add_collection3d(tri) + + ax.set(xlim=(0, 1), ylim=(0, 1), zlim=(0, 1), + xticks=(1,), yticks=(1,), zticks=(1,)) + + ax.set_xticklabels(['$(1, 0, 0)$'], fontsize=12) + ax.set_yticklabels(['$(0, 1, 0)$'], fontsize=12) + ax.set_zticklabels(['$(0, 0, 1)$'], fontsize=12) + + ax.xaxis.majorTicks[0].set_pad(15) + ax.yaxis.majorTicks[0].set_pad(15) + ax.zaxis.majorTicks[0].set_pad(35) + + ax.view_init(30, angle) + + # Move axis to origin + ax.xaxis._axinfo['juggled'] = (0, 0, 0) + ax.yaxis._axinfo['juggled'] = (1, 1, 1) + ax.zaxis._axinfo['juggled'] = (2, 2, 0) + + ax.grid(False) + + return ax + +Q = np.array(Q) +ψ_00 = np.array((0.01, 0.01, 0.99)) +ψ_01 = np.array((0.01, 0.99, 0.01)) +ψ_02 = np.array((0.99, 0.01, 0.01)) + +ax = unit_simplex(angle=50) + +def flow_plot(ψ, h=0.001, n=300, angle=50): + colors = cm.jet_r(np.linspace(0.0, 1, n)) + + x_vals, y_vals, z_vals = [], [], [] + for t in range(n): + x_vals.append(ψ[0]) + y_vals.append(ψ[1]) + z_vals.append(ψ[2]) + ψ = ψ @ expm(h * Q) + + ax.scatter(x_vals, y_vals, z_vals, c=colors, s=20, alpha=0.2, depthshade=False) + +flow_plot(ψ_00) +flow_plot(ψ_01) +flow_plot(ψ_02) + +# Add stationary distribution +P_1 = expm(Q) +mc = qe.MarkovChain(P_1) +ψ = mc.stationary_distributions[0] +ax.scatter(ψ[0], ψ[1], ψ[2], c='k', s=30, depthshade=False) + +plt.show() +``` + +This black dot is the stationary distribution $\psi^*$ of the Markov semigroup $(P_t)$ generated by $Q$. + +It was calculated the ``stationary_distributions`` attribute of QuantEcon's +[``MarkovChain`` class](https://quanteconpy.readthedocs.io/en/latest/markov.html), by arbitrarily setting $t=1$ and solving $\psi P_1 = \psi$. + +Below we show that, for this choice of $Q$, the stationary distribution +$\psi^*$ is unique in $\dD$, due to irreducibility. + +Moreover, $\psi P_t \to \psi^*$ as $t \to \infty$ for any $\psi \in \dD$, as +suggested by the figure. + + -The next result makes this possible. +### Stationarity via the Generator -It is analogous to the idea that a point $\bar x$ in $\RR^d$ is stationary for +In many cases, it is easier to use the generator of the semigroup to identify +stationary distributions rather than the semigroup itself. + +This is analogous to the idea that a point $\bar x$ in $\RR^d$ is stationary for a vector ODE $x'_t = F(x_t)$ when $F(\bar x) = 0$. -It holds true under weaker conditions but the version stated here is easy to -prove and sufficient for most cases we consider. +(Here $F$ is the infinitesimal description, and hence analogous to the generator.) + +The next result holds true under weaker conditions but the version stated here +is easy to prove and sufficient for applications we consider. + ```{proof:theorem} :label: statfromq -For a conservative intensity matrix $Q$ on $S$ and -associated Markov semigroup $(P_t)$, a distribution $\psi$ on $S$ is stationary -for $(P_t)$ if and only if $\psi Q = 0$. +Let $(P_t)$ be a UC Markov semigroup with generator $Q$. A distribution +$\psi$ on $S$ is stationary for $(P_t)$ if and only if $\psi Q = 0$. ``` ```{proof:proof} -Suppose first that $\psi \in \dD$ and $\psi Q = 0$. +Fix $\psi \in \dD$ and suppose first that $\psi Q = 0$. -Since $Q$ is conservative, $(P_t)$ is a UC semigroup with $P_t = e^{tQ}$ for -all $t$, and hence, for any $t \geq 0$, +Since $(P_t)$ is a UC Markov semigroup, we have $P_t = e^{tQ}$ for all $t$, +and hence, for any $t \geq 0$, $$ \psi e^{tQ} = \psi + t \psi Q + t^2 \frac{\psi Q^2}{2!} @@ -99,11 +227,11 @@ $$ $$ Since $(P_t)$ is UC and $Q$ is its generator, we have $\| D_t - Q \| \to 0$ in -$\lL(\ell_1)$ as $t \to 0$ from the right. +$\lL(\ell_1)$ as $t \to 0$. Hence $\| \psi Q \| \leq \liminf_{t \to 0} \| \psi D_t \|$. -As $\psi$ is stationary, $\psi D_t = 0$ for all $t$. +As $\psi$ is stationary for $(P_t)$, we have $\psi D_t = 0$ for all $t$. Hence $\psi Q = 0$, as was to be shown. ``` @@ -117,36 +245,45 @@ Let $(P_t)$ be a Markov semigroup on $S$. We say that state $y$ is **accessible** from state $x$ if there exists a $t \geq 0$ such that $P_t(x, y) > 0$. -A Markov semigroup $(P_t)$ on $S$ is called **irreducible** if $y$ is -accessible from $x$ for every $(x,y) \in S \times S$. +We say that $x$ and $y$ **communicate** if $x$ is accessible from $y$ and $y$ +is accessible from $x$. + +A Markov semigroup $(P_t)$ on $S$ is called **irreducible** if +every pair $x,y$ in $S$ communicates. We seek a characterization of irreducibility of $(P_t)$ in terms of its generator. As a first step, we will say there is a **$Q$-positive probability flow** from $x$ -to $y$ if there exists a finite sequence $(z_i)_{i=1}^m$ in $S$ starting at -$x=z_1$ and ending at $y=z_m$ such that $Q(z_i, z_{i+1}) > 0$ for all $i$. +to $y$ if there exists a finite sequence $(z_i)_{i=0}^m$ in $S$ starting at +$x=z_0$ and ending at $y=z_m$ such that $Q(z_i, z_{i+1}) > 0$ for all $i$. -```{proof:lemma} +```{proof:theorem} :label: equivirr +Let $(P_t)$ be a UC Markov semigroup with generator $Q$. For distinct states $x$ and $y$, the following statements are equivalent: 1. The state $y$ is accessible from $x$ under $(P_t)$. -1. There is a positive probability flow from $x$ to $y$ under $Q$. +1. There is a $Q$-positive probability flow from $x$ to $y$. +1. $P_t(x, y) > 0$ for all $t > 0$. ``` -((prove only for the UC case?)) ```{proof:proof} -For distinct states $x$ and $y$, we have +Pick any two distinct states $x$ and $y$. + +It is obvious that statement 3 implies statement 1, so we need only prove +(1 $\implies$ 2) and (2 $\implies$ 3). + +Starting with (1 $\implies$ 2), recall that $$ P_t(x, y) = t Q(x,y) + \frac{t^2}{2!} Q^2(x, y) + \cdots $$ (ptexpan) -If $x$ is accessible from $y$, then $P_t(x, y) > 0$ for some $t > 0$, and hence +If $x$ is accessible from $y$, then $P_t(x, y) > 0$ for some $t > 0$, so $Q^k(x,y) > 0$ for at least one $k \in \NN$. Writing out the matrix product as a sum, we now have @@ -161,53 +298,403 @@ $$ > 0 $$ (qkassum) -It follows that at least one element of the sum must be strictly positive, so -positive probability flow from $x$ to $y$ is established. +It follows that at least one element of the sum must be strictly positive. -To prove the converse, suppose there is positive probability flow from $x$ to -$y$. +Therefore, a $Q$-positive probability flow from $x$ to $y$ exists. -We can then take a finite sequence $(z_i)_{i=1}^m$ starting at $x$ and ending -at $y$ with $Q(z_i, z_{i+1}) > 0$ for all $i$. +Turning to (2 $\implies$ 3), first note that, for arbitrary states $u$ and $v$, if $Q(u, v) > 0$ then $P_t(u, v) > 0$ for all $t > 0$. -We can and do assume that $(z_i)_{i=0}^m$ is the shortest such sequence -(otherwise just pick the shortest one). +To see this, let $(\lambda, K)$ be the jump chain pair constructed from $Q$ via +{eq}`lambdafromq`, {eq}`kfromqxx` and {eq}`kfromqxy`. -Because any shorter sequence $(u_i)_{i=1}^k$ linking $x$ and $y$ has $Q(u_i, u_{i+1}) = 0$ for some $i$, {eq}`qkassum` implies that $Q^k(x,y) = 0$ for all $k < m$. +Observe that, since $Q(u, v) > 0$, we must have $\lambda(u) > 0$. -Hence, by {eq}`ptexpan`, we have $P_t(x, y) = t^m Q^m(x, y) + o(t^m)$. +As a consequence, applying {eq}`kfromqxy`, we have -As $Q^m(x, y) > 0$, we see that $P_t(x, y) > 0$ for sufficiently small $t$. +$$ + K(u, v) = \frac{Q(u, v)}{\lambda(u)} > 0 +$$ -``` -Irreducible implies aperiodic. +Let $(Y_k)$ and $(J_k)$ be the embedded jump chain and jump sequence generated +by {proof:ref}`ejc_algo`, with $Y_0 = u$. -```{proof:theorem} -For a ((UC)) Markov semigroup $(P_t)$, the following statements are +With $E_1 \sim \Exp(1)$ and $E_2 \sim \Exp(1)$, we have, for any $t > 0$, + +$$ +\begin{aligned} + P_t(u, v) + & \geq \PP \{J_1 \leq t, \, Y_1 = v, \, J_2 > t \} + \\ + & \geq \PP \{E_1 \leq t\lambda(u), \, E_2 > t\lambda(v) \} \PP\{ Y_1 = v \} + \\ + & = \PP \{E_1 \leq t\lambda(u)\} + \PP \{ E_2 > t\lambda(v) \} K(u, v) + \\ + & > 0 +\end{aligned} +$$ + +Now suppose there is a $Q$-positive probability flow $(z_i)_{i=0}^m$ from $x$ +to $y$. + +If we fix $t > 0$ and repeatedly apply {eq}`chapkol_ct2` along with the last +result, we obtain + +$$ + P_t(x, y) + \geq + \prod_{i=0}^{m-1} P_{t/m} (z_i, z_{i+1}) > 0 +$$ + + + +``` + +{proof:ref}`equivirr` leads directly to the following strong result. + +```{proof:corollary} +:label: perimposs +For a UC Markov semigroup $(P_t)$, the following statements are equivalent: 1. $(P_t)$ is irreducible. 1. $P_t(x,y) > 0$ for all $t > 0$ and all $(x,y) \in S \times S$. ``` + +```{note} +To obtain stable long run behavior in discrete time Markov chains, it is +common to assume that the chain is aperiodic. + +This needs to be assumed on top of irreducibility if one wishes to rule out +all dependence on initial conditions. + +{proof:ref}`perimposs` shows that periodicity is not a concern for irreducible +continuous time Markov chains. + +Positive probability flow from $x$ to $y$ at some $t > 0$ immediately implies +positive flow for all $t > 0$. + +``` + + + + +## Asymptotic Stabiltiy + +We call Markov semigroup $(P_t)$ **asymptotically stable** if $(P_t)$ has a +unique stationary distribution $\psi^*$ in $\dD$ and + +$$ + \| \psi P_t - \psi^* \| \to 0 \text{ as } t \to \infty + \text{ for all } \psi \in \dD +$$ (asyms) + +Our aim is to establish conditions for asymptotic stability of Markov +semigroups. + + + +### Contractivity + +Let's recall some useful facts about the discrete time case. + +First, if $P$ is any Markov matrix, we have, in the $\ell_1$ norm, + +$$ + \| \psi P \| \leq \| \psi \| + \text{ for all } \psi \in \dD +$$ (allmocontract) + +This is because, given $\psi \in \dD$, + +$$ + \| \psi P \| + = \sum_y \left| \sum_x \psi(x) P(x, y) \right| + \leq \sum_y \sum_x | \psi(x) | P(x, y) + = \| \psi \| +$$ + +By linearity, for $\psi, \phi \in \dD$, we then have + +$$ + \| \psi P - \phi P \| \leq \| \psi - \phi \| +$$ + +Hence every Markov operator is contracting on $\dD$. + +Moreover, if $P$ is everywhere positive, then this inequality is strict: + +```{proof:lemma} Strict Contractivity +:label: strictcontract + +If $P$ is a Markov matrix and $P(x, y) > 0$ for all $x, y$, then + +$$ + \| \psi P - \phi P \| < \| \psi - \phi \| + \text{ for all } \psi, \phi \in \dD + \text{ with } \psi \not= \phi +$$ + +``` + +The proof follows from the strict triangle inequality, as opposed to the weak +triangle inequality we used to obtain {eq}`allmocontract`. + +See, for example, Proposition 3.1.2 of {cite}`lasota1994chaos` or Lemma 8.2.3 of {cite}`stachurski2009economic`. + +### Uniqueness + +Irreducibility of a given Markov chain implies that there are no disjoint +absorbing sets. + +This in turn leads to uniqueness of stationary distributions: + +```{proof:theorem} +:label: uniirr + +Let $(P_t)$ be a UC Markov semigroup on $S$. If $(P_t)$ is irreducible, then +$(P_t)$ has at most one stationary distribution. +``` + + ```{proof:proof} +Suppose to the contrary that $\psi$ and $\phi$ are both stationary for +$(P_t)$. + +Since $(P_t)$ is irreducible, we know that $P_1(x,y) > 0$ for all $x,y \in S$. -To be added. +If $\psi \not= \phi$, then, due to positivity of $P_1$, the strict inequality +in {proof:ref}`strictcontract` holds. + +At the same time, by stationarity, $\| \psi P - \phi P \| = \| \psi - \phi +\|$. Contradiction. ``` -Irreducible implies uniqueness. +```{proof:example} +An M/M/1 queue with parameters $\mu, \lambda$ is a continuous time Markov chain $(X_t)$ on $S = \ZZ_+$ with with intensity matrix +$$ +Q=\begin{pmatrix} + -\lambda & \lambda & 0 & 0 & \cdots \\ + \mu & -(\mu + \lambda) & \lambda & 0 & \cdots \\ + 0 & \mu & -(\mu + \lambda) & \lambda & \cdots\\ + \vdots & \vdots & \vdots & \vdots & \ddots +\end{pmatrix} +$$ (mm1q) -## Asymptotic Stabiltiy +The chain $(X_t)$ records the length of the queue at each moment in time. + +The intensity matrix captures the idea that customers flow into the queue at +rate $\lambda$ and are served (and hence leave the queue) at rate $\mu$. + +If $\lambda$ and $\mu$ are both positive, then there is a $Q$-positive +probability flow between any two states, in both directions, so the +corresponding semigroup $(P_t)$ is irreducible. + +{proof:ref}`uniirr` now tells us that $(P_t)$ has at most one stationary +distribution. + +``` + + +### Stability from the Skeleton + +Recall the definition of asymptotic stability given in {eq}`asyms`. + +Analogously, we call an individual Markov operator $P$ asymptotically stable if +$P$ has a unique stationary distribution $\psi^*$ in $\dD$ and +$\psi P^n \to \psi^*$ as $n \to \infty$ for all $\psi \in \dD$. + +The next result gives a connection between discrete and continuous stability. + +The critical ingredient linking these two concepts is the contractivity in +{eq}`allmocontract`. + +```{proof:lemma} +:label: stabskel + +Let $(P_t)$ be a Markov semigroup. If there exists an $s > 0$ such that the +Markov matrix $P_s$ is asymptotically stable, then $(P_t)$ is asymptotically +stable with the same stationary distribution. + +``` + +```{proof:proof} + +Let $(P_t)$ and $s$ be as in the statement of {proof:ref}`stabskel`. + + +Let $\psi^*$ be the stationary distribution of $P_s$. Fix $\psi \in \dD$ and +$\epsilon > 0$. + +By stability of $P_s$, we can take an $n \in \NN$ such that +$\| \psi P_s^n - \psi^* \| < \epsilon$. + +Pick any $t > sn$. Set $h := t - sn$. + +By the contractivity in {eq}`allmocontract` and $P_{sn} = P_s^n$, we have + + +$$ + \| \psi P_t - \psi^* \| + = \| \psi P_{sn} P_h - \psi^* P_h \| + \leq \| \psi P_{sn} - \psi^* \| + < \epsilon. +$$ + +Hence asymptotic stability holds + +``` + + +### Stability via Drift + +In this section we address drift conditions, which are a powerful method for +obtaining asymptotic stability when the state space can be infinite. + +The idea is to show that the state tends to drift back to a finite set over +time. + +Such drift, when combined with the contractivity in +{proof:ref}`strictcontract`, is enough to give global stability. + +The next theorem gives a useful version of this class of results. + +```{proof:theorem} +:label: sdrift + +Let $(P_t)$ be a UC Markov semigroup with intensity matrix $Q$. If $(P_t)$ is +irreducible and there exists a function $v \colon S \to \RR_+$, a finite set +$F \subset S$ and positive constants $\epsilon$ and $M$ such that + +$$ + \sum_y Q(x, y) v(y) + \leq + \begin{cases} + M & \text{ if } x \in F \text{ and } + \\ + - \epsilon & \text{ otherwise } + \end{cases} +$$ + +then $(P_t)$ is asymptotically stable. +``` + +The proof of {proof:ref}`sdrift` can be found in {cite}`pichor2012stochastic`. + +```{proof:example} + +Consider again the M/M/1 queue on $\ZZ_+$ with intensity matrix {eq}`mm1q`. + +Suppose that $\lambda < \mu$. + +It is intuitive that, in this case, the queue length will not +tend to infinity (since the service rate is higher than the arrival rate). + +This intuition can be confirmed via {proof:ref}`sdrift`, after setting $v(i) = +i$. -To be added. +Indeed, we have, for any $i \geq 1$, -Include Foguel alternative? +$$ + \sum_{j \geq 0} Q(i, j) v(j) + = (i-1) \mu - i (\mu + \lambda) + (i+1) \lambda + = \lambda - \mu +$$ + +Setting $F=\{0\}$ and $\epsilon = \mu - \lambda$, we see that the conditions +of {proof:ref}`sdrift` hold. + +Hence the associated semigroup $(P_t)$ is asymptotically stable. + +``` -Thm 6.2 of RR and MTK. +```{proof:corollary} +:label: sfinite + +If $(P_t)$ is an irreducible UC Markov semigroup and $S$ is finite, then +$(P_t)$ is asymptotically stable. +``` + +A solved exercise below asks you to confirm this. + +## Exercises + +### Exercise 1 + +Let $(P_t)$ be a Markov semigroup. True or false: +for this semigroup, every state $x$ is accessible from itself. + + +### Exercise 2 + +Let $(\lambda_k)$ be a bounded sequence in $(0, \infty)$. + +A **pure birth process** starting at zero is a continuous time Markov process +$(X_t)$ on state space $\ZZ_+$ with intensity matrix + +$$ +Q=\begin{pmatrix} + -\lambda_0 & \lambda_0 & 0 & 0 & \cdots \\ + 0 & -\lambda_1 & \lambda_1 & 0 & \cdots \\ + 0 & 0 & -\lambda_2 & \lambda_2 & \cdots\\ + \vdots & \vdots & \vdots & \vdots & \ddots +\end{pmatrix} +$$ + +Show that $(P_t)$, the corresponding Markov semigroup, has no stationary +distribution. + +### Exercise 3 + +Confirm that {proof:ref}`sdrift` implies {proof:ref}`sfinite`. + + +## Solutions + +### Solution to Exercise 1 + +The statement is true. With $t=0$ we have $P_t(x,x) = I(x,x) = 1 > 0$. + +### Solution to Exercise 2 + +Suppose to the contrary that $\phi \in \dD$ and $\phi Q = 0$. + +Then + +$$ + (\phi Q)(j) + = \sum_{i \geq 0} \phi(i) Q(i, j) + = - \lambda_j \phi(j) + \lambda_j \phi(j+1) + = 0 +$$ + +It follows that $\phi$ is constant on $\ZZ_+$. + +But $\dD$ contains no constant functions when the state space is infinite. +(Why?) + +Contradiction. + +### Solution to Exercise 3 + +Let $(P_t)$ be an irreducible UC Markov semigroup and let $S$ be finite. + +Pick any positive constants $M, \epsilon$ and set $v = M$ and $F = S$. + +We then have + +$$ + \sum_y Q(x, y) v(y) + = M \sum_y Q(x, y) + = 0 +$$ +Hence the drift condition in {proof:ref}`sdrift` holds and $(P_t)$ is +asymptotically stable. diff --git a/_sources/intro.md b/_sources/intro.md index 1945286..2d0be0f 100644 --- a/_sources/intro.md +++ b/_sources/intro.md @@ -1,7 +1,54 @@ Continuous Time Markov Chains ============================= -Introductory text to be added. +**Authors**: [Thomas J. Sargent](http://www.tomsargent.com/) and [John +Stachurski](https://johnstachurski.net/) -Uses --- list applications. +This lecture series provides a short introduction to the +fascinating field of continuous time Markov chains. It will, in time, be +integrated into our [QuantEcon lectures](https://quantecon.org/python-lectures/). +Focus is shared between theory, applications and computation. Mathematical +ideas are combined with computer code to help clarify and build intuition, as +well as to bridge the gap between theory and applications. +The presentation is relatively rigorous but the aim is towards applications +rather than mathematical curiosities (which are plentiful, if one starts to +look). Applications are mainly drawn from economics and operations research. + +```{admonition} Solved exercises + +There are many solved exercises and we recommend readers attempt all of them, +or at least review the solutions. + +``` + + +```{admonition} Computer code + +The code is written in Python and is accelerated through a combination of +[NumPy](https://numpy.org/) (vectorized code) and just-in-time compilation +(via [Numba](http://numba.pydata.org/)). +QuantEcon provides a fast-paced [introduction to scientific computing with Python](https://python-programming.quantecon.org/) that covers these topics. +``` + + + +```{admonition} Background: Markov chains in discrete time + +The lectures are well suited to those who have some knowledge of discrete time +Markov chains and wish to learn more about their continuous time cousins. A +suitable preliminary discussion of discrete time Markov chains can be found +[here](https://python.quantecon.org/finite_markov.html). + +``` + +```{admonition} Prerequisites: Probability and Analysis + +Readers are assumed to be familiar with probability and a small amount of +analysis. Later lectures, which deal with infinite state spaces, assume that +require that readers are comfortable with the basics of linear analysis in Banach space. + +``` + + +The lectures are written using [Jupyter Book](https://jupyterbook.org/intro.html). diff --git a/_sources/kolmogorov_bwd.ipynb b/_sources/kolmogorov_bwd.ipynb index 06f5b55..71c90d7 100644 --- a/_sources/kolmogorov_bwd.ipynb +++ b/_sources/kolmogorov_bwd.ipynb @@ -91,7 +91,7 @@ "When we relax it, the jump intensities depend on the state.\n", "\n", "(jumpchainalgo)=\n", - "### Embedded Jump Chain Algorithm\n", + "### Jump Chain Algorithm\n", "\n", "We start with three primitives\n", "\n", @@ -110,7 +110,7 @@ "\n", "Here is the same algorithm written more explicitly: \n", "\n", - "```{proof:algorithm} Embedded Jump Chain Algorithm\n", + "```{proof:algorithm} Jump Chain Algorithm\n", ":label: ejc_algo\n", "\n", "**Inputs** $\\psi \\in \\dD$, rate function $\\lambda$, Markov matrix $K$\n", diff --git a/_sources/kolmogorov_bwd.md b/_sources/kolmogorov_bwd.md index c8c1fe4..77a8562 100644 --- a/_sources/kolmogorov_bwd.md +++ b/_sources/kolmogorov_bwd.md @@ -89,7 +89,7 @@ This assumption was made purely for convenience and seems unlikely to hold true. When we relax it, the jump intensities depend on the state. (jumpchainalgo)= -### Embedded Jump Chain Algorithm +### Jump Chain Algorithm We start with three primitives @@ -108,7 +108,7 @@ Now we take $y$ as the new state for the process and repeat. Here is the same algorithm written more explicitly: -```{proof:algorithm} Embedded Jump Chain Algorithm +```{proof:algorithm} Jump Chain Algorithm :label: ejc_algo **Inputs** $\psi \in \dD$, rate function $\lambda$, Markov matrix $K$ diff --git a/_sources/kolmogorov_fwd.ipynb b/_sources/kolmogorov_fwd.ipynb index 9b8e3ca..ed385e8 100644 --- a/_sources/kolmogorov_fwd.ipynb +++ b/_sources/kolmogorov_fwd.ipynb @@ -87,7 +87,7 @@ "\n", "where distributions are understood as row vectors.\n", "\n", - "Here's a visualization for the case $|S|=3$, so that $\\dD$ is the unit\n", + "Here's a visualization for the case $S = \\{0, 1, 2\\}$, so that $\\dD$ is the unit\n", "simplex in $\\RR^3$.\n", "\n", "The initial condition is `` (0, 0, 1)`` and the Markov matrix is" @@ -259,6 +259,7 @@ " \\end{pmatrix}\n", "$$\n", "\n", + "(solvode)=\n", "### Solutions to Linear Vector ODEs\n", "\n", "Using the matrix exponential, the unique solution to the initial value problem\n", @@ -733,9 +734,9 @@ 93, 99, 163, - 271, - 277, - 304 + 272, + 278, + 305 ] }, "nbformat": 4, diff --git a/_sources/kolmogorov_fwd.md b/_sources/kolmogorov_fwd.md index e02ceff..d59e5c6 100644 --- a/_sources/kolmogorov_fwd.md +++ b/_sources/kolmogorov_fwd.md @@ -85,7 +85,7 @@ $$ where distributions are understood as row vectors. -Here's a visualization for the case $|S|=3$, so that $\dD$ is the unit +Here's a visualization for the case $S = \{0, 1, 2\}$, so that $\dD$ is the unit simplex in $\RR^3$. The initial condition is `` (0, 0, 1)`` and the Markov matrix is @@ -224,6 +224,7 @@ $$ \end{pmatrix} $$ +(solvode)= ### Solutions to Linear Vector ODEs Using the matrix exponential, the unique solution to the initial value problem diff --git a/_sources/prob_view.ipynb b/_sources/prob_view.ipynb deleted file mode 100644 index df7cc32..0000000 --- a/_sources/prob_view.ipynb +++ /dev/null @@ -1,270 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# A Probabilistic View\n", - "\n", - "\n", - "## Overview\n", - "\n", - "Let $S$ be a countable state space and let $Q$ be a conservative intensity\n", - "matrix on $S$.\n", - "\n", - "We know that $Q$ generates a UC Markov semigroup, and from this semigroup we\n", - "can produce a continuous time Markov chain via the associated joint distribution.\n", - "\n", - "This method of building chains from intensity matrices is important from a\n", - "theoretical perspective but less helpful in terms of intuition and simulation.\n", - "\n", - "In this lecture we provide another point of view.\n", - "\n", - "In particular, we provide two different ways to build a Markov chain with\n", - "intensity matrix $Q$.\n", - "\n", - "One is via jump chains with state dependent jump rates.\n", - "\n", - "The second is another probabilistic construction, sometimes called Gillespie's\n", - "Algorithm.\n", - "\n", - "These two constructions provide intuition from multiple perspectives and\n", - "valuable simulation algorithms.\n", - "\n", - "We will use the following imports" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [], - "source": [ - "import numpy as np\n", - "import scipy as sp\n", - "import matplotlib.pyplot as plt\n", - "import quantecon as qe\n", - "from numba import njit\n", - "from scipy.linalg import expm" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## From Intensity Matrix to Jump Chain\n", - "\n", - "Let us agree to call $(\\lambda, K)$ a **jump chain pair** if $\\lambda$ is a\n", - "map from $S$ to $\\RR_+$ and $K$ is a Markov matrix on $S$.\n", - "\n", - "It is easy to verify that the matrix $Q$ on $S$ defined by \n", - "\n", - "$$\n", - " Q(x, y) := \\lambda(x) (K(x, y) - I(x, y))\n", - "$$ (jcinmat)\n", - "\n", - "is an intensity matrix.\n", - "\n", - "(We saw in {doc}`an earlier lecture ` that $Q$ is the intensity\n", - "matrix for the jump chain $(X_t)$ built via {proof:ref}`ejc_algo` from jump\n", - "chain pair $(\\lambda, K)$.)\n", - "\n", - "As we now show, every intensity matrix admits the decomposition in\n", - "{eq}`jcinmat` for some jump chain pair.\n", - "\n", - "\n", - "(constlk)=\n", - "### Construction \n", - "\n", - "Given a intensity matrix $Q$, set \n", - "\n", - "$$\n", - " \\lambda(x) := -Q(x, x)\n", - " \\qquad (x \\in S)\n", - "$$\n", - "\n", - "Next we build $K$, first along the principle diagonal via\n", - "\n", - "$$\n", - " K(x,x) = \n", - " \\begin{cases}\n", - " 0 & \\text{ if } \\lambda(x) > 0\n", - " \\\\\n", - " 1 & \\text{ otherwise}\n", - " \\end{cases}\n", - "$$\n", - "\n", - "Thus, if the rate of leaving $x$ is positive, we set $K(x,x) = 0$, so that the\n", - "embedded jump chain moves away from $x$ with probability one when the next\n", - "jump occurs.\n", - "\n", - "Otherwise, when $Q(x,x) = 0$, we stay at $x$ forever, so $x$ is an **absorbing\n", - "state**.\n", - "\n", - "Off the principle diagonal, where $x \\not= y$, we set\n", - "\n", - "$$\n", - " K(x,y) = \n", - " \\begin{cases}\n", - " \\frac{Q(x,y)}{\\lambda(x)} & \\text{ if } \\lambda(x) > 0\n", - " \\\\\n", - " 0 & \\text{ otherwise }\n", - " \\end{cases}\n", - "$$\n", - "\n", - "The exercises below ask you to confirm that, for $\\lambda$ and $K$ just defined, \n", - "\n", - "1. $(\\lambda, K)$ is a jump chain pair and\n", - "1. the intensity matrix $Q$ satisfies {eq}`jcinmat`.\n", - "\n", - "We call $(\\lambda, K)$ the **jump chain decomposition** of $Q$.\n", - "\n", - "We summarize in a lemma.\n", - "\n", - "```{proof:lemma}\n", - ":label: imatjc\n", - "\n", - "A matrix $Q$ on $S$ is an intensity matrix if and only if there exists a jump\n", - "chain pair $(\\lambda, K)$ such that {eq}`jcinmat` holds.\n", - "```\n", - "\n", - "\n", - "### The Conservative Case\n", - "\n", - "\n", - "We know from {proof:ref}`jccs` that an intensity matrix $Q$ is conservative\n", - "if and only if $\\lambda$ is bounded.\n", - "\n", - "Moreover, we saw in {proof:ref}`usmg` that the pairing between conservative\n", - "intensity matrices and UC Markov semigroups is one-to-one.\n", - "\n", - "This leads to the following result.\n", - "\n", - "```{proof:theorem}\n", - "On $S$, there exists a one-to-one correspondence between the following sets of\n", - "objects:\n", - "\n", - "1. The set of all jump chain pairs $(\\lambda, K)$ such that $\\lambda$ is bounded.\n", - "1. The set of all conservative intensity matrices.\n", - "1. The set of all UC Markov semigroups.\n", - "\n", - "```\n", - "\n", - "\n", - "\n", - "### Simulation\n", - "\n", - "\n", - "In view of the preceding discussion, we have a simple way to simulate a Markov\n", - "chain given any conservative intensity matrix $Q$.\n", - "\n", - "The steps are\n", - "\n", - "1. Decompose $Q$ into a jump chain pair $(\\lambda, K)$.\n", - "2. Simulate via {proof:ref}`ejc_algo`.\n", - "\n", - "\n", - "Recalling our discussion of the Kolmogorov backward equation, we know that \n", - "this produces a Markov chain with Markov semigroup\n", - "$(P_t)$ where $P_t = e^{tQ}$ for $Q$ satisfying {eq}`jcinmat`.\n", - "\n", - "(Although our argument assumed finite $S$, the proof goes through when\n", - "$S$ is contably infinite and $Q$ is conservative with very minor changes.)\n", - "\n", - "In particular, $(X_t)$ is a continuous time Markov chain with intensity matrix\n", - "$Q$.\n", - "\n", - "\n", - "\n", - "## The Gillespie Algorithm\n", - "\n", - "Exponential clocks.\n", - "\n", - "Show by simulation that distributions coincide with $\\psi_0 P_t$.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Exercises\n", - "\n", - "### Exercise 1\n", - "\n", - "Let $Q$ be any intensity matrix on $S$.\n", - "\n", - "Prove that the jump chain decomposition of $Q$ is in fact a jump chain pair.\n", - "\n", - "Prove that, in addition, this decomposition $(\\lambda, K)$ satisfies {eq}`jcinmat`.\n", - "\n", - "## Solutions\n", - "\n", - "### Solution to Exercise 1\n", - "\n", - "Let $Q$ be an intensity matrix and let $(\\lambda, K)$ be the jump chain\n", - "decomposition of $Q$.\n", - "\n", - "Nonnegativity of $\\lambda$ is immediate from the definition of an intensity\n", - "matrix.\n", - "\n", - "To see that $K$ is a Markov matrix we fix $x \\in S$ and suppose first that \n", - "$\\lambda(x) > 0$.\n", - "\n", - "Then \n", - "\n", - "$$\n", - " \\sum_y K(x, y) \n", - " = \\sum_{y \\not= x} K(x,y) \n", - " = \\sum_{y \\not= x} \\frac{Q(x,y)}{\\lambda(x)}\n", - " = 1\n", - "$$\n", - "\n", - "If, on the other hand, $\\lambda(x) = 0$, then \n", - "$\\sum_y K(x, y) = 1$, is immediate from the definition.\n", - "\n", - "As $K$ is nonnegative, we see that $K$ is a Markov matrix.\n", - "\n", - "Thus, $(\\lambda, K)$ is a valid jump chain pair.\n", - "\n", - "The proof that $Q$ and $(\\lambda, K)$ satisfy {eq}`jcinmat` is mechanical and\n", - "the details are omitted.\n", - "\n", - "(Try working case-by-case, with $\\lambda(x) = 0, x=y$, $\\lambda(x) > 0, x=y$, etc.)" - ] - } - ], - "metadata": { - "jupytext": { - "formats": "ipynb,md:myst", - "text_representation": { - "extension": ".md", - "format_name": "myst", - "format_version": "0.9", - "jupytext_version": "1.5.0" - } - }, - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.7.7" - }, - "source_map": [ - 13, - 45, - 52 - ] - }, - "nbformat": 4, - "nbformat_minor": 4 -} \ No newline at end of file diff --git a/_sources/prob_view.md b/_sources/prob_view.md deleted file mode 100644 index 1d014d4..0000000 --- a/_sources/prob_view.md +++ /dev/null @@ -1,231 +0,0 @@ ---- -jupytext: - formats: ipynb,md:myst - text_representation: - extension: .md - format_name: myst - format_version: '0.9' - jupytext_version: 1.5.0 -kernelspec: - display_name: Python 3 - language: python - name: python3 ---- - - -# A Probabilistic View - - -## Overview - -Let $S$ be a countable state space and let $Q$ be a conservative intensity -matrix on $S$. - -We know that $Q$ generates a UC Markov semigroup, and from this semigroup we -can produce a continuous time Markov chain via the associated joint distribution. - -This method of building chains from intensity matrices is important from a -theoretical perspective but less helpful in terms of intuition and simulation. - -In this lecture we provide another point of view. - -In particular, we provide two different ways to build a Markov chain with -intensity matrix $Q$. - -One is via jump chains with state dependent jump rates. - -The second is another probabilistic construction, sometimes called Gillespie's -Algorithm. - -These two constructions provide intuition from multiple perspectives and -valuable simulation algorithms. - -We will use the following imports - -```{code-cell} ipython3 -import numpy as np -import scipy as sp -import matplotlib.pyplot as plt -import quantecon as qe -from numba import njit -from scipy.linalg import expm -``` - - -## From Intensity Matrix to Jump Chain - -Let us agree to call $(\lambda, K)$ a **jump chain pair** if $\lambda$ is a -map from $S$ to $\RR_+$ and $K$ is a Markov matrix on $S$. - -It is easy to verify that the matrix $Q$ on $S$ defined by - -$$ - Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) -$$ (jcinmat) - -is an intensity matrix. - -(We saw in {doc}`an earlier lecture ` that $Q$ is the intensity -matrix for the jump chain $(X_t)$ built via {proof:ref}`ejc_algo` from jump -chain pair $(\lambda, K)$.) - -As we now show, every intensity matrix admits the decomposition in -{eq}`jcinmat` for some jump chain pair. - - -(constlk)= -### Construction - -Given a intensity matrix $Q$, set - -$$ - \lambda(x) := -Q(x, x) - \qquad (x \in S) -$$ - -Next we build $K$, first along the principle diagonal via - -$$ - K(x,x) = - \begin{cases} - 0 & \text{ if } \lambda(x) > 0 - \\ - 1 & \text{ otherwise} - \end{cases} -$$ - -Thus, if the rate of leaving $x$ is positive, we set $K(x,x) = 0$, so that the -embedded jump chain moves away from $x$ with probability one when the next -jump occurs. - -Otherwise, when $Q(x,x) = 0$, we stay at $x$ forever, so $x$ is an **absorbing -state**. - -Off the principle diagonal, where $x \not= y$, we set - -$$ - K(x,y) = - \begin{cases} - \frac{Q(x,y)}{\lambda(x)} & \text{ if } \lambda(x) > 0 - \\ - 0 & \text{ otherwise } - \end{cases} -$$ - -The exercises below ask you to confirm that, for $\lambda$ and $K$ just defined, - -1. $(\lambda, K)$ is a jump chain pair and -1. the intensity matrix $Q$ satisfies {eq}`jcinmat`. - -We call $(\lambda, K)$ the **jump chain decomposition** of $Q$. - -We summarize in a lemma. - -```{proof:lemma} -:label: imatjc - -A matrix $Q$ on $S$ is an intensity matrix if and only if there exists a jump -chain pair $(\lambda, K)$ such that {eq}`jcinmat` holds. -``` - - -### The Conservative Case - - -We know from {proof:ref}`jccs` that an intensity matrix $Q$ is conservative -if and only if $\lambda$ is bounded. - -Moreover, we saw in {proof:ref}`usmg` that the pairing between conservative -intensity matrices and UC Markov semigroups is one-to-one. - -This leads to the following result. - -```{proof:theorem} -On $S$, there exists a one-to-one correspondence between the following sets of -objects: - -1. The set of all jump chain pairs $(\lambda, K)$ such that $\lambda$ is bounded. -1. The set of all conservative intensity matrices. -1. The set of all UC Markov semigroups. - -``` - - - -### Simulation - - -In view of the preceding discussion, we have a simple way to simulate a Markov -chain given any conservative intensity matrix $Q$. - -The steps are - -1. Decompose $Q$ into a jump chain pair $(\lambda, K)$. -2. Simulate via {proof:ref}`ejc_algo`. - - -Recalling our discussion of the Kolmogorov backward equation, we know that -this produces a Markov chain with Markov semigroup -$(P_t)$ where $P_t = e^{tQ}$ for $Q$ satisfying {eq}`jcinmat`. - -(Although our argument assumed finite $S$, the proof goes through when -$S$ is contably infinite and $Q$ is conservative with very minor changes.) - -In particular, $(X_t)$ is a continuous time Markov chain with intensity matrix -$Q$. - - - -## The Gillespie Algorithm - -Exponential clocks. - -Show by simulation that distributions coincide with $\psi_0 P_t$. - - - - - -## Exercises - -### Exercise 1 - -Let $Q$ be any intensity matrix on $S$. - -Prove that the jump chain decomposition of $Q$ is in fact a jump chain pair. - -Prove that, in addition, this decomposition $(\lambda, K)$ satisfies {eq}`jcinmat`. - -## Solutions - -### Solution to Exercise 1 - -Let $Q$ be an intensity matrix and let $(\lambda, K)$ be the jump chain -decomposition of $Q$. - -Nonnegativity of $\lambda$ is immediate from the definition of an intensity -matrix. - -To see that $K$ is a Markov matrix we fix $x \in S$ and suppose first that -$\lambda(x) > 0$. - -Then - -$$ - \sum_y K(x, y) - = \sum_{y \not= x} K(x,y) - = \sum_{y \not= x} \frac{Q(x,y)}{\lambda(x)} - = 1 -$$ - -If, on the other hand, $\lambda(x) = 0$, then -$\sum_y K(x, y) = 1$, is immediate from the definition. - -As $K$ is nonnegative, we see that $K$ is a Markov matrix. - -Thus, $(\lambda, K)$ is a valid jump chain pair. - -The proof that $Q$ and $(\lambda, K)$ satisfy {eq}`jcinmat` is mechanical and -the details are omitted. - -(Try working case-by-case, with $\lambda(x) = 0, x=y$, $\lambda(x) > 0, x=y$, etc.) diff --git a/_sources/uc_mc_semigroups.ipynb b/_sources/uc_mc_semigroups.ipynb index 6078ebb..28f08f8 100644 --- a/_sources/uc_mc_semigroups.ipynb +++ b/_sources/uc_mc_semigroups.ipynb @@ -8,19 +8,26 @@ "\n", "## Overview\n", "\n", - "In our previous lecture we covered some of the general theory of continuous\n", - "operator semigroups. \n", + "In our previous lecture we covered some of the general theory of operator\n", + "semigroups. \n", + "\n", + "\n", "\n", "Next we translate these results into the setting of Markov semigroups.\n", "\n", - "The main aim is to give an exact one-to-one correspondence between UC\n", - "Markov semigroups and \"conservative\" intensity matrices on a countable set $S$.\n", + "The Markov semigroups are defined on a countable set $S$.\n", + "\n", + "The main aim is to give an exact one-to-one correspondence between \n", + "\n", + "* UC Markov semigroups \n", + "* \"conservative\" intensity matrices and\n", + "* jump chains with state dependent jump intensities\n", "\n", "Conservativeness is defined below and relates to \"nonexplosiveness\" of the\n", - "associated Markov matrix.\n", + "associated Markov chain.\n", "\n", - "We will also give a brief discussion of intensity matricies that fall\n", - "outside this class, along with the processes they generate.\n", + "We will also give a brief discussion of intensity matricies that do not have\n", + "this property, along with the processes they generate.\n", "\n", "\n", "## Notation and Terminology\n", @@ -81,7 +88,7 @@ "\n", "\n", "\n", - "## A Unique Pairing\n", + "## UC Markov Semigroups and their Generators\n", "\n", "Let $Q$ be a conservative intensity matrix on $S$.\n", "\n", @@ -308,6 +315,143 @@ "\n", "\n", "\n", + "## From Intensity Matrix to Jump Chain\n", + "\n", + "We now understand that there is a one-to-one pairing between conservative\n", + "intenintensity matrices and UC Markov semigroups.\n", + "\n", + "These ideas are important from an analytical perspective.\n", + "\n", + "Now we provide another point of view, more connected to probability.\n", + "\n", + "This point of view is important for both theory and computation.\n", + "\n", + "\n", + "\n", + "### Jump Chain Pairs\n", + "\n", + "Let us agree to call $(\\lambda, K)$ a **jump chain pair** if $\\lambda$ is a\n", + "map from $S$ to $\\RR_+$ and $K$ is a Markov matrix on $S$.\n", + "\n", + "It is easy to verify that the matrix $Q$ on $S$ defined by \n", + "\n", + "$$\n", + " Q(x, y) := \\lambda(x) (K(x, y) - I(x, y))\n", + "$$ (jcinmat)\n", + "\n", + "is an intensity matrix.\n", + "\n", + "(We saw in {doc}`an earlier lecture ` that $Q$ is the intensity\n", + "matrix for the jump chain $(X_t)$ built via {proof:ref}`ejc_algo` from jump\n", + "chain pair $(\\lambda, K)$.)\n", + "\n", + "As we now show, every intensity matrix admits the decomposition in\n", + "{eq}`jcinmat` for some jump chain pair.\n", + "\n", + "\n", + "### Jump Chain Decomposition\n", + "\n", + "Given a intensity matrix $Q$, set \n", + "\n", + "$$\n", + " \\lambda(x) := -Q(x, x)\n", + " \\qquad (x \\in S)\n", + "$$ (lambdafromq)\n", + "\n", + "Next we build $K$, first along the principle diagonal via\n", + "\n", + "$$\n", + " K(x,x) = \n", + " \\begin{cases}\n", + " 0 & \\text{ if } \\lambda(x) > 0\n", + " \\\\\n", + " 1 & \\text{ otherwise}\n", + " \\end{cases}\n", + "$$ (kfromqxx)\n", + "\n", + "Thus, if the rate of leaving $x$ is positive, we set $K(x,x) = 0$, so that the\n", + "embedded jump chain moves away from $x$ with probability one when the next\n", + "jump occurs.\n", + "\n", + "Otherwise, when $Q(x,x) = 0$, we stay at $x$ forever, so $x$ is an **absorbing\n", + "state**.\n", + "\n", + "Off the principle diagonal, where $x \\not= y$, we set\n", + "\n", + "$$\n", + " K(x,y) = \n", + " \\begin{cases}\n", + " \\frac{Q(x,y)}{\\lambda(x)} & \\text{ if } \\lambda(x) > 0\n", + " \\\\\n", + " 0 & \\text{ otherwise }\n", + " \\end{cases}\n", + "$$ (kfromqxy)\n", + "\n", + "The exercises below ask you to confirm that, for $\\lambda$ and $K$ just defined, \n", + "\n", + "1. $(\\lambda, K)$ is a jump chain pair and\n", + "1. the intensity matrix $Q$ satisfies {eq}`jcinmat`.\n", + "\n", + "We call $(\\lambda, K)$ the **jump chain decomposition** of $Q$.\n", + "\n", + "We summarize in a lemma.\n", + "\n", + "```{proof:lemma}\n", + ":label: imatjc\n", + "\n", + "A matrix $Q$ on $S$ is an intensity matrix if and only if there exists a jump\n", + "chain pair $(\\lambda, K)$ such that {eq}`jcinmat` holds.\n", + "```\n", + "\n", + "\n", + "### The Conservative Case\n", + "\n", + "\n", + "We know from {proof:ref}`jccs` that an intensity matrix $Q$ is conservative\n", + "if and only if $\\lambda$ is bounded.\n", + "\n", + "Moreover, we saw in {proof:ref}`usmg` that the pairing between conservative\n", + "intensity matrices and UC Markov semigroups is one-to-one.\n", + "\n", + "This leads to the following result.\n", + "\n", + "```{proof:theorem}\n", + "On $S$, there exists a one-to-one correspondence between the following sets of\n", + "objects:\n", + "\n", + "1. The set of all jump chain pairs $(\\lambda, K)$ such that $\\lambda$ is bounded.\n", + "1. The set of all conservative intensity matrices.\n", + "1. The set of all UC Markov semigroups.\n", + "\n", + "```\n", + "\n", + "\n", + "\n", + "### Simulation\n", + "\n", + "\n", + "In view of the preceding discussion, we have a simple way to simulate a Markov\n", + "chain given any conservative intensity matrix $Q$.\n", + "\n", + "The steps are\n", + "\n", + "1. Decompose $Q$ into a jump chain pair $(\\lambda, K)$.\n", + "2. Simulate via {proof:ref}`ejc_algo`.\n", + "\n", + "\n", + "Recalling our discussion of the Kolmogorov backward equation, we know that \n", + "this produces a Markov chain with Markov semigroup\n", + "$(P_t)$ where $P_t = e^{tQ}$ for $Q$ satisfying {eq}`jcinmat`.\n", + "\n", + "(Although our argument assumed finite $S$, the proof goes through when\n", + "$S$ is contably infinite and $Q$ is conservative with very minor changes.)\n", + "\n", + "In particular, $(X_t)$ is a continuous time Markov chain with intensity matrix\n", + "$Q$.\n", + "\n", + "\n", + "\n", + "\n", "## Beyond Bounded Intensity Matrices\n", "\n", "If we do run into an application where an intensity matrix $Q$ is not\n", @@ -368,6 +512,17 @@ "Show that, with $K^m$ representing the $m$-th matrix product of $K$ with itself, \n", "we have $K^m(i, j) = \\mathbb 1\\{j = i + m\\}$ for any $i, j \\in \\ZZ_+$.\n", " \n", + "### Exercise 5\n", + "\n", + "Let $Q$ be any intensity matrix on $S$.\n", + "\n", + "Prove that the jump chain decomposition of $Q$ is in fact a jump chain pair.\n", + "\n", + "Prove that, in addition, this decomposition $(\\lambda, K)$ satisfies {eq}`jcinmat`.\n", + "\n", + "\n", + "\n", + "\n", "\n", "## Solutions\n", "\n", @@ -479,7 +634,40 @@ " = K(i, m-j)\n", "$$\n", "\n", - "Applying the definition $K(i, j) = \\mathbb 1\\{j = i + 1\\}$ completes verification of the claim." + "Applying the definition $K(i, j) = \\mathbb 1\\{j = i + 1\\}$ completes verification of the claim.\n", + "\n", + "\n", + "### Solution to Exercise 5\n", + "\n", + "Let $Q$ be an intensity matrix and let $(\\lambda, K)$ be the jump chain\n", + "decomposition of $Q$.\n", + "\n", + "Nonnegativity of $\\lambda$ is immediate from the definition of an intensity\n", + "matrix.\n", + "\n", + "To see that $K$ is a Markov matrix we fix $x \\in S$ and suppose first that \n", + "$\\lambda(x) > 0$.\n", + "\n", + "Then \n", + "\n", + "$$\n", + " \\sum_y K(x, y) \n", + " = \\sum_{y \\not= x} K(x,y) \n", + " = \\sum_{y \\not= x} \\frac{Q(x,y)}{\\lambda(x)}\n", + " = 1\n", + "$$\n", + "\n", + "If, on the other hand, $\\lambda(x) = 0$, then \n", + "$\\sum_y K(x, y) = 1$, is immediate from the definition.\n", + "\n", + "As $K$ is nonnegative, we see that $K$ is a Markov matrix.\n", + "\n", + "Thus, $(\\lambda, K)$ is a valid jump chain pair.\n", + "\n", + "The proof that $Q$ and $(\\lambda, K)$ satisfy {eq}`jcinmat` is mechanical and\n", + "the details are omitted.\n", + "\n", + "(Try working case-by-case, with $\\lambda(x) = 0, x=y$, $\\lambda(x) > 0, x=y$, etc.)" ] } ], diff --git a/_sources/uc_mc_semigroups.md b/_sources/uc_mc_semigroups.md index d60aed0..d674bdf 100644 --- a/_sources/uc_mc_semigroups.md +++ b/_sources/uc_mc_semigroups.md @@ -17,19 +17,26 @@ kernelspec: ## Overview -In our previous lecture we covered some of the general theory of continuous -operator semigroups. +In our previous lecture we covered some of the general theory of operator +semigroups. + + Next we translate these results into the setting of Markov semigroups. -The main aim is to give an exact one-to-one correspondence between UC -Markov semigroups and "conservative" intensity matrices on a countable set $S$. +The Markov semigroups are defined on a countable set $S$. + +The main aim is to give an exact one-to-one correspondence between + +* UC Markov semigroups +* "conservative" intensity matrices and +* jump chains with state dependent jump intensities Conservativeness is defined below and relates to "nonexplosiveness" of the -associated Markov matrix. +associated Markov chain. -We will also give a brief discussion of intensity matricies that fall -outside this class, along with the processes they generate. +We will also give a brief discussion of intensity matricies that do not have +this property, along with the processes they generate. ## Notation and Terminology @@ -90,7 +97,7 @@ Below we show how this property can be checked in applications. -## A Unique Pairing +## UC Markov Semigroups and their Generators Let $Q$ be a conservative intensity matrix on $S$. @@ -317,6 +324,143 @@ intensity matrices. +## From Intensity Matrix to Jump Chain + +We now understand that there is a one-to-one pairing between conservative +intenintensity matrices and UC Markov semigroups. + +These ideas are important from an analytical perspective. + +Now we provide another point of view, more connected to probability. + +This point of view is important for both theory and computation. + + + +### Jump Chain Pairs + +Let us agree to call $(\lambda, K)$ a **jump chain pair** if $\lambda$ is a +map from $S$ to $\RR_+$ and $K$ is a Markov matrix on $S$. + +It is easy to verify that the matrix $Q$ on $S$ defined by + +$$ + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +$$ (jcinmat) + +is an intensity matrix. + +(We saw in {doc}`an earlier lecture ` that $Q$ is the intensity +matrix for the jump chain $(X_t)$ built via {proof:ref}`ejc_algo` from jump +chain pair $(\lambda, K)$.) + +As we now show, every intensity matrix admits the decomposition in +{eq}`jcinmat` for some jump chain pair. + + +### Jump Chain Decomposition + +Given a intensity matrix $Q$, set + +$$ + \lambda(x) := -Q(x, x) + \qquad (x \in S) +$$ (lambdafromq) + +Next we build $K$, first along the principle diagonal via + +$$ + K(x,x) = + \begin{cases} + 0 & \text{ if } \lambda(x) > 0 + \\ + 1 & \text{ otherwise} + \end{cases} +$$ (kfromqxx) + +Thus, if the rate of leaving $x$ is positive, we set $K(x,x) = 0$, so that the +embedded jump chain moves away from $x$ with probability one when the next +jump occurs. + +Otherwise, when $Q(x,x) = 0$, we stay at $x$ forever, so $x$ is an **absorbing +state**. + +Off the principle diagonal, where $x \not= y$, we set + +$$ + K(x,y) = + \begin{cases} + \frac{Q(x,y)}{\lambda(x)} & \text{ if } \lambda(x) > 0 + \\ + 0 & \text{ otherwise } + \end{cases} +$$ (kfromqxy) + +The exercises below ask you to confirm that, for $\lambda$ and $K$ just defined, + +1. $(\lambda, K)$ is a jump chain pair and +1. the intensity matrix $Q$ satisfies {eq}`jcinmat`. + +We call $(\lambda, K)$ the **jump chain decomposition** of $Q$. + +We summarize in a lemma. + +```{proof:lemma} +:label: imatjc + +A matrix $Q$ on $S$ is an intensity matrix if and only if there exists a jump +chain pair $(\lambda, K)$ such that {eq}`jcinmat` holds. +``` + + +### The Conservative Case + + +We know from {proof:ref}`jccs` that an intensity matrix $Q$ is conservative +if and only if $\lambda$ is bounded. + +Moreover, we saw in {proof:ref}`usmg` that the pairing between conservative +intensity matrices and UC Markov semigroups is one-to-one. + +This leads to the following result. + +```{proof:theorem} +On $S$, there exists a one-to-one correspondence between the following sets of +objects: + +1. The set of all jump chain pairs $(\lambda, K)$ such that $\lambda$ is bounded. +1. The set of all conservative intensity matrices. +1. The set of all UC Markov semigroups. + +``` + + + +### Simulation + + +In view of the preceding discussion, we have a simple way to simulate a Markov +chain given any conservative intensity matrix $Q$. + +The steps are + +1. Decompose $Q$ into a jump chain pair $(\lambda, K)$. +2. Simulate via {proof:ref}`ejc_algo`. + + +Recalling our discussion of the Kolmogorov backward equation, we know that +this produces a Markov chain with Markov semigroup +$(P_t)$ where $P_t = e^{tQ}$ for $Q$ satisfying {eq}`jcinmat`. + +(Although our argument assumed finite $S$, the proof goes through when +$S$ is contably infinite and $Q$ is conservative with very minor changes.) + +In particular, $(X_t)$ is a continuous time Markov chain with intensity matrix +$Q$. + + + + ## Beyond Bounded Intensity Matrices If we do run into an application where an intensity matrix $Q$ is not @@ -377,6 +521,17 @@ Let $K$ be defined on $\ZZ_+ \times \ZZ_+$ by $K(i, j) = \mathbb 1\{j = i + 1\}$ Show that, with $K^m$ representing the $m$-th matrix product of $K$ with itself, we have $K^m(i, j) = \mathbb 1\{j = i + m\}$ for any $i, j \in \ZZ_+$. +### Exercise 5 + +Let $Q$ be any intensity matrix on $S$. + +Prove that the jump chain decomposition of $Q$ is in fact a jump chain pair. + +Prove that, in addition, this decomposition $(\lambda, K)$ satisfies {eq}`jcinmat`. + + + + ## Solutions @@ -491,3 +646,36 @@ $$ Applying the definition $K(i, j) = \mathbb 1\{j = i + 1\}$ completes verification of the claim. +### Solution to Exercise 5 + +Let $Q$ be an intensity matrix and let $(\lambda, K)$ be the jump chain +decomposition of $Q$. + +Nonnegativity of $\lambda$ is immediate from the definition of an intensity +matrix. + +To see that $K$ is a Markov matrix we fix $x \in S$ and suppose first that +$\lambda(x) > 0$. + +Then + +$$ + \sum_y K(x, y) + = \sum_{y \not= x} K(x,y) + = \sum_{y \not= x} \frac{Q(x,y)}{\lambda(x)} + = 1 +$$ + +If, on the other hand, $\lambda(x) = 0$, then +$\sum_y K(x, y) = 1$, is immediate from the definition. + +As $K$ is nonnegative, we see that $K$ is a Markov matrix. + +Thus, $(\lambda, K)$ is a valid jump chain pair. + +The proof that $Q$ and $(\lambda, K)$ satisfy {eq}`jcinmat` is mechanical and +the details are omitted. + +(Try working case-by-case, with $\lambda(x) = 0, x=y$, $\lambda(x) > 0, x=y$, etc.) + + diff --git a/_static/jupyter-sphinx.css b/_static/jupyter-sphinx.css deleted file mode 100644 index cea8405..0000000 --- a/_static/jupyter-sphinx.css +++ /dev/null @@ -1,116 +0,0 @@ -/* Stylesheet for jupyter-sphinx - -These styles mimic the Jupyter HTML styles. - -The default CSS (Cascading Style Sheet) class structure of jupyter-sphinx -is the following: - -jupyter_container - code_cell (optional) - stderr (optional) - output (optional) - -If the code_cell is not displayed, then there is not a jupyter_container, and -the output is provided without CSS. - -This stylesheet attempts to override the defaults of all packaged Sphinx themes -to display jupter-sphinx cells in a Jupyter-like style. - -If you want to adjust the styles, add additional custom CSS to override these -styles. - -After a build, this stylesheet is loaded from ./_static/jupyter-sphinx.css . - -*/ - - -div.jupyter_container { - padding: .4em; - margin: 0 0 .4em 0; - background-color: #FFFF; - border: 1px solid #CCC; - -moz-box-shadow: 2px 2px 4px rgba(87, 87, 87, 0.2); - -webkit-box-shadow: 2px 2px 4px rgba(87, 87, 87, 0.2); - box-shadow: 2px 2px 4px rgba(87, 87, 87, 0.2); -} -.jupyter_container div.code_cell { - border: 1px solid #cfcfcf; - border-radius: 2px; - background-color: #f7f7f7; - margin: 0 0; - overflow: auto; -} - -.jupyter_container div.code_cell pre { - padding: 4px; - margin: 0 0; - background-color: #f7f7f7; - border: none; - background: none; - box-shadow: none; - -webkit-box-shadow: none; /* for nature */ - -moz-box-shadow: none; /* for nature */ -} - -.jupyter_container div.code_cell * { - margin: 0 0; -} -div.jupyter_container div.highlight { - background-color: #f7f7f7; /* for haiku */ -} -div.jupyter_container { - padding: 0; - margin: 0; -} -/* overrides for sphinx_rtd_theme */ -.rst-content .jupyter_container div[class^='highlight'], -.document .jupyter_container div[class^='highlight'], -.rst-content .jupyter_container pre.literal-block { - border:none; - margin: 0; - padding: 0; - background: none; - padding: 3px; - background-color: transparent; -} -/* restore Mathjax CSS, as it assumes a vertical margin. */ -.jupyter_container .MathJax_Display { - margin: 1em 0em; - text-align: center; -} -.jupyter_container .stderr { - background-color: #FCC; - border: none; - padding: 3px; -} -.jupyter_container .output { - border: none; -} -.jupyter_container div.output pre { - background-color: white; - background: none; - padding: 4px; - border: none; - box-shadow: none; - -webkit-box-shadow: none; /* for nature */ - -moz-box-shadow: none; /* for nature */ -} -.jupyter_container .code_cell td.linenos { - text-align: right; - padding: 4px 4px 4px 8px; - border-right: 1px solid #cfcfcf; - color: #999; -} -.jupyter_container .output .highlight { - background-color: #ffffff; -} -/* combine sequential jupyter cells, - by moving sequential ones up higher on y-axis */ -div.jupyter_container + div.jupyter_container { - margin: -.5em 0 .4em 0; -} - -/* Fix for sphinx_rtd_theme spacing after jupyter_container #91 */ -.rst-content .jupyter_container { - margin: 0 0 24px 0; -} diff --git a/_static/mystnb.js b/_static/mystnb.js deleted file mode 100644 index cc4f800..0000000 --- a/_static/mystnb.js +++ /dev/null @@ -1,44 +0,0 @@ -// Initialize MathJax with the notebook config -// Taken from https://github.com/jupyter/notebook/blob/master/notebook/static/notebook/js/mathjaxutils.js#L13 -var initMathJax = () => { - if (window.MathJax) { - // MathJax loaded - MathJax.Hub.Config({ - tex2jax: { - inlineMath: [ ['$','$'], ["\\(","\\)"] ], - displayMath: [ ['$$','$$'], ["\\[","\\]"] ], - processEscapes: true, - processEnvironments: true - }, - MathML: { - extensions: ['content-mathml.js'] - }, - // Center justify equations in code and markdown cells. Elsewhere - // we use CSS to left justify single line equations in code cells. - displayAlign: 'center', - "HTML-CSS": { - availableFonts: [], - imageFont: null, - preferredFont: null, - webFont: "STIX-Web", - styles: {'.MathJax_Display': {"margin": 0}}, - linebreaks: { automatic: true } - }, - }); - MathJax.Hub.Configured(); - } -} - -// Helper function to run when the DOM is finished -const mystNBRunWhenDOMLoaded = cb => { - if (document.readyState != 'loading') { - cb() - } else if (document.addEventListener) { - document.addEventListener('DOMContentLoaded', cb) - } else { - document.attachEvent('onreadystatechange', function() { - if (document.readyState == 'complete') cb() - }) - } - } -mystNBRunWhenDOMLoaded(initMathJax) diff --git a/ergodicity.html b/ergodicity.html index 4243e75..0fb7f47 100644 --- a/ergodicity.html +++ b/ergodicity.html @@ -39,7 +39,7 @@ - + @@ -113,11 +113,6 @@

Continuous Time Markov Chains

UC Markov Semigroups -
  • - - A Probabilistic View - -
  • Stationarity and Ergodicity @@ -197,7 +192,7 @@

    Continuous Time Markov Chains

    aria-label="Launch interactive content">
    -
  • @@ -237,6 +244,72 @@

    Continuous Time Markov Chains

     Asymptotic Stabiltiy + +
    • +
  • + + Exercises + + +
    • +
  • + + Solutions + +
    • @@ -252,10 +325,25 @@

    Continuous Time Markov Chains

    Stationarity and Ergodicity

    Overview

    -

    To be added.

    -

    Use the distribution flow figures from markov_prop.md, this time starting from -many different distributions.

    -

    Suggests ergodicity.

    +

    In this lecture we discuss stability and equilibrium behavior for continuous +time Markov chains.

    +

    To give one example of why this theory matters, consider queues, which are +often modeled as continuous time Markov chains.

    +

    Queueing theory is used in applications such as

    +
      +

    • treatment of patients arriving at a hospital

      • +

    • optimal design of manufacturing processes

      • +

    • requests to a file server

      • +

    • air traffic

      • +

    • customers waiting on a help line

      • +
    +

    A key topic in queueing theory is average behavior over the long run.

    +
      +

    • Will the length of the queue grow without bounds?

      • +

    • If not, is there some kind of long run equilibrium?

      • +

    • If so, what is the average waiting time in this equilibrium?

      • +

    • What is the average length of the queue over a week, or a month?

      • +

    We will use the following imports

    @@ -264,7 +352,9 @@

    Overviewimport matplotlib.pyplot as plt import quantecon as qe from numba import njit +from scipy.linalg import expm +from matplotlib import cm from mpl_toolkits.mplot3d import Axes3D from mpl_toolkits.mplot3d.art3d import Poly3DCollection

    @@ -274,31 +364,135 @@

    Overview

    Stationary Distributions

    -

    Let \((P_t)\) be a Markov semigroup on countable state space \(S\).

    -

    A distribution \(\psi \in \dD\) is called stationary for \((P_t)\) if

    +
    +

    Definition

    +

    Let \(S\) be countable.

    +

    Recall that, for a discrete time Markov chain with Markov matrix \(P\) on \(S\), a +distribution \(\psi\) is called stationary for \(P\) if \(\psi P = \psi\).

    +

    This means that if \(X_t\) has distribution \(\psi\), then so does \(X_{t+1}\).

    +

    For continuous time Markov chains, the definition is analogous.

    +

    Given a Markov semigroup on \(S\), a distribution \(\psi^* \in \dD\) is called +stationary for \((P_t)\) if

    \[ - \psi P_t = \psi + \psi^* P_t = \psi^* \text{ for all } t \geq 0 \]
    +

    As one example, we look again at the chain on \(S = \{0, 1, +2\}\) with intensity matrix

    +
    +
    +
    Q = ((-3, 2, 1),
+     (3, -5, 2),
+     (4, 6, -10))
+
    +
    +
    +
    +

    The following figure was shown before, except that now there is a black dot +that the three trajectories seem to be converging to.

    +

    (Recall that, in the color scheme, trajectories cool as time evolves.)

    +
    +
    +
    def unit_simplex(angle):
+    
+    fig = plt.figure(figsize=(8, 6))
+    ax = fig.add_subplot(111, projection='3d')
+
+    vtx = [[0, 0, 1],
+           [0, 1, 0], 
+           [1, 0, 0]]
+    
+    tri = Poly3DCollection([vtx], color='darkblue', alpha=0.3)
+    tri.set_facecolor([0.5, 0.5, 1])
+    ax.add_collection3d(tri)
+
+    ax.set(xlim=(0, 1), ylim=(0, 1), zlim=(0, 1), 
+           xticks=(1,), yticks=(1,), zticks=(1,))
+
+    ax.set_xticklabels(['$(1, 0, 0)$'], fontsize=12)
+    ax.set_yticklabels(['$(0, 1, 0)$'], fontsize=12)
+    ax.set_zticklabels(['$(0, 0, 1)$'], fontsize=12)
+
+    ax.xaxis.majorTicks[0].set_pad(15)
+    ax.yaxis.majorTicks[0].set_pad(15)
+    ax.zaxis.majorTicks[0].set_pad(35)
+
+    ax.view_init(30, angle)
+
+    # Move axis to origin
+    ax.xaxis._axinfo['juggled'] = (0, 0, 0)
+    ax.yaxis._axinfo['juggled'] = (1, 1, 1)
+    ax.zaxis._axinfo['juggled'] = (2, 2, 0)
+    
+    ax.grid(False)
+    
+    return ax
+
+Q = np.array(Q)
+ψ_00 = np.array((0.01, 0.01, 0.99))
+ψ_01 = np.array((0.01, 0.99, 0.01))
+ψ_02 = np.array((0.99, 0.01, 0.01))
+
+ax = unit_simplex(angle=50)    
+
+def flow_plot(ψ, h=0.001, n=300, angle=50):
+    colors = cm.jet_r(np.linspace(0.0, 1, n))
+
+    x_vals, y_vals, z_vals = [], [], []
+    for t in range(n):
+        x_vals.append(ψ[0])
+        y_vals.append(ψ[1])
+        z_vals.append(ψ[2])
+        ψ = ψ @ expm(h * Q)
+
+    ax.scatter(x_vals, y_vals, z_vals, c=colors, s=20, alpha=0.2, depthshade=False)
+
+flow_plot(ψ_00)
+flow_plot(ψ_01)
+flow_plot(ψ_02)
+
+# Add stationary distribution
+P_1 = expm(Q)
+mc = qe.MarkovChain(P_1)
+ψ = mc.stationary_distributions[0]
+ax.scatter(ψ[0], ψ[1], ψ[2], c='k', s=30, depthshade=False)
+
+plt.show()
+
    +
    +
    +
    +_images/ergodicity_5_0.png +
    +
    +

    This black dot is the stationary distribution \(\psi^*\) of the Markov semigroup \((P_t)\) generated by \(Q\).

    +

    It was calculated the stationary_distributions attribute of QuantEcon’s +MarkovChain class, by arbitrarily setting \(t=1\) and solving \(\psi P_1 = \psi\).

    +

    Below we show that, for this choice of \(Q\), the stationary distribution +\(\psi^*\) is unique in \(\dD\), due to irreducibility.

    +

    Moreover, \(\psi P_t \to \psi^*\) as \(t \to \infty\) for any \(\psi \in \dD\), as +suggested by the figure.

    +
    +
    +

    Stationarity via the Generator

    In many cases, it is easier to use the generator of the semigroup to identify -stationary distributions.

    -

    The next result makes this possible.

    -

    It is analogous to the idea that a point \(\bar x\) in \(\RR^d\) is stationary for +stationary distributions rather than the semigroup itself.

    +

    This is analogous to the idea that a point \(\bar x\) in \(\RR^d\) is stationary for a vector ODE \(x'_t = F(x_t)\) when \(F(\bar x) = 0\).

    -

    It holds true under weaker conditions but the version stated here is easy to -prove and sufficient for most cases we consider.

    +

    (Here \(F\) is the infinitesimal description, and hence analogous to the generator.)

    +

    The next result holds true under weaker conditions but the version stated here +is easy to prove and sufficient for applications we consider.

    Theorem 23

    -

    For a conservative intensity matrix \(Q\) on \(S\) and -associated Markov semigroup \((P_t)\), a distribution \(\psi\) on \(S\) is stationary -for \((P_t)\) if and only if \(\psi Q = 0\).

    +

    Let \((P_t)\) be a UC Markov semigroup with generator \(Q\). A distribution +\(\psi\) on \(S\) is stationary for \((P_t)\) if and only if \(\psi Q = 0\).

    -

    Proof. Suppose first that \(\psi \in \dD\) and \(\psi Q = 0\).

    -

    Since \(Q\) is conservative, \((P_t)\) is a UC semigroup with \(P_t = e^{tQ}\) for -all \(t\), and hence, for any \(t \geq 0\),

    +

    Proof. Fix \(\psi \in \dD\) and suppose first that \(\psi Q = 0\).

    +

    Since \((P_t)\) is a UC Markov semigroup, we have \(P_t = e^{tQ}\) for all \(t\), +and hence, for any \(t \geq 0\),

    \[ \psi e^{tQ} = \psi + t \psi Q + t^2 \frac{\psi Q^2}{2!} @@ -317,45 +511,52 @@

    Stationary Distributions\((P_t)\) is UC and \(Q\) is its generator, we have \(\| D_t - Q \| \to 0\) in -\(\lL(\ell_1)\) as \(t \to 0\) from the right.

    +\(\lL(\ell_1)\) as \(t \to 0\).

    Hence \(\| \psi Q \| \leq \liminf_{t \to 0} \| \psi D_t \|\).

    -

    As \(\psi\) is stationary, \(\psi D_t = 0\) for all \(t\).

    +

    As \(\psi\) is stationary for \((P_t)\), we have \(\psi D_t = 0\) for all \(t\).

    Hence \(\psi Q = 0\), as was to be shown.

    +

    Irreducibility and Uniqueness

    Let \((P_t)\) be a Markov semigroup on \(S\).

    We say that state \(y\) is accessible from state \(x\) if there exists a \(t \geq 0\) such that \(P_t(x, y) > 0\).

    -

    A Markov semigroup \((P_t)\) on \(S\) is called irreducible if \(y\) is -accessible from \(x\) for every \((x,y) \in S \times S\).

    +

    We say that \(x\) and \(y\) communicate if \(x\) is accessible from \(y\) and \(y\) +is accessible from \(x\).

    +

    A Markov semigroup \((P_t)\) on \(S\) is called irreducible if +every pair \(x,y\) in \(S\) communicates.

    We seek a characterization of irreducibility of \((P_t)\) in terms of its generator.

    As a first step, we will say there is a \(Q\)-positive probability flow from \(x\) -to \(y\) if there exists a finite sequence \((z_i)_{i=1}^m\) in \(S\) starting at -\(x=z_1\) and ending at \(y=z_m\) such that \(Q(z_i, z_{i+1}) > 0\) for all \(i\).

    -
    -

    Lemma 24

    -
    -

    For distinct states \(x\) and \(y\), the following statements are equivalent:

    +to \(y\) if there exists a finite sequence \((z_i)_{i=0}^m\) in \(S\) starting at +\(x=z_0\) and ending at \(y=z_m\) such that \(Q(z_i, z_{i+1}) > 0\) for all \(i\).

    +
    +

    Theorem 24

    +
    +

    Let \((P_t)\) be a UC Markov semigroup with generator \(Q\). +For distinct states \(x\) and \(y\), the following statements are equivalent:

    1. The state \(y\) is accessible from \(x\) under \((P_t)\).

      2. -

    2. There is a positive probability flow from \(x\) to \(y\) under \(Q\).

      3. +

    3. There is a \(Q\)-positive probability flow from \(x\) to \(y\).

      4. +

    4. \(P_t(x, y) > 0\) for all \(t > 0\).

    -

    ((prove only for the UC case?))

    -
    -

    Proof. For distinct states \(x\) and \(y\), we have

    +
    +

    Proof. Pick any two distinct states \(x\) and \(y\).

    +

    It is obvious that statement 3 implies statement 1, so we need only prove +(1 \(\implies\) 2) and (2 \(\implies\) 3).

    +

    Starting with (1 \(\implies\) 2), recall that

    -(54)\[ +(57)\[ P_t(x, y) = t Q(x,y) + \frac{t^2}{2!} Q^2(x, y) + \cdots \]
    -

    If \(x\) is accessible from \(y\), then \(P_t(x, y) > 0\) for some \(t > 0\), and hence +

    If \(x\) is accessible from \(y\), then \(P_t(x, y) > 0\) for some \(t > 0\), so \(Q^k(x,y) > 0\) for at least one \(k \in \NN\).

    Writing out the matrix product as a sum, we now have

    -

    Irreducible implies aperiodic.

    -
    -

    Theorem 25

    -
    -

    For a ((UC)) Markov semigroup \((P_t)\), the following statements are +

    It follows that at least one element of the sum must be strictly positive.

    +

    Therefore, a \(Q\)-positive probability flow from \(x\) to \(y\) exists.

    +

    Turning to (2 \(\implies\) 3), first note that, for arbitrary states \(u\) and \(v\), if \(Q(u, v) > 0\) then \(P_t(u, v) > 0\) for all \(t > 0\).

    +

    To see this, let \((\lambda, K)\) be the jump chain pair constructed from \(Q\) via +(54), (55) and (56).

    +

    Observe that, since \(Q(u, v) > 0\), we must have \(\lambda(u) > 0\).

    +

    As a consequence, applying (56), we have

    +
    +\[ + K(u, v) = \frac{Q(u, v)}{\lambda(u)} > 0 +\]
    +

    Let \((Y_k)\) and \((J_k)\) be the embedded jump chain and jump sequence generated +by Algorithm 6, with \(Y_0 = u\).

    +

    With \(E_1 \sim \Exp(1)\) and \(E_2 \sim \Exp(1)\), we have, for any \(t > 0\),

    +
    +\[\begin{split} +\begin{aligned} + P_t(u, v) + & \geq \PP \{J_1 \leq t, \, Y_1 = v, \, J_2 > t \} + \\ + & \geq \PP \{E_1 \leq t\lambda(u), \, E_2 > t\lambda(v) \} \PP\{ Y_1 = v \} + \\ + & = \PP \{E_1 \leq t\lambda(u)\} + \PP \{ E_2 > t\lambda(v) \} K(u, v) + \\ + & > 0 +\end{aligned} +\end{split}\]
    +

    Now suppose there is a \(Q\)-positive probability flow \((z_i)_{i=0}^m\) from \(x\) +to \(y\).

    +

    If we fix \(t > 0\) and repeatedly apply (15) along with the last +result, we obtain

    +
    +\[ + P_t(x, y) + \geq + \prod_{i=0}^{m-1} P_{t/m} (z_i, z_{i+1}) > 0 +\]
    +
    +

    Theorem 24 leads directly to the following strong result.

    +
    +

    Corollary 25

    +
    +

    For a UC Markov semigroup \((P_t)\), the following statements are equivalent:

    1. \((P_t)\) is irreducible.

    2. \(P_t(x,y) > 0\) for all \(t > 0\) and all \((x,y) \in S \times S\).

    -
    -

    Proof. To be added.

    +
    +

    Note

    +

    To obtain stable long run behavior in discrete time Markov chains, it is +common to assume that the chain is aperiodic.

    +

    This needs to be assumed on top of irreducibility if one wishes to rule out +all dependence on initial conditions.

    +

    Corollary 25 shows that periodicity is not a concern for irreducible +continuous time Markov chains.

    +

    Positive probability flow from \(x\) to \(y\) at some \(t > 0\) immediately implies +positive flow for all \(t > 0\).

    -

    Irreducible implies uniqueness.

    Asymptotic Stabiltiy

    -

    To be added.

    -

    Include Foguel alternative?

    -

    Thm 6.2 of RR and MTK.

    +

    We call Markov semigroup \((P_t)\) asymptotically stable if \((P_t)\) has a +unique stationary distribution \(\psi^*\) in \(\dD\) and

    +
    +(59)\[ + \| \psi P_t - \psi^* \| \to 0 \text{ as } t \to \infty + \text{ for all } \psi \in \dD +\]
    +

    Our aim is to establish conditions for asymptotic stability of Markov +semigroups.

    +
    +

    Contractivity

    +

    Let’s recall some useful facts about the discrete time case.

    +

    First, if \(P\) is any Markov matrix, we have, in the \(\ell_1\) norm,

    +
    +(60)\[ + \| \psi P \| \leq \| \psi \| + \text{ for all } \psi \in \dD +\]
    +

    This is because, given \(\psi \in \dD\),

    +
    +\[ + \| \psi P \| + = \sum_y \left| \sum_x \psi(x) P(x, y) \right| + \leq \sum_y \sum_x | \psi(x) | P(x, y) + = \| \psi \| +\]
    +

    By linearity, for \(\psi, \phi \in \dD\), we then have

    +
    +\[ + \| \psi P - \phi P \| \leq \| \psi - \phi \| +\]
    +

    Hence every Markov operator is contracting on \(\dD\).

    +

    Moreover, if \(P\) is everywhere positive, then this inequality is strict:

    +
    +

    Lemma 26 (Strict Contractivity)

    +
    +

    If \(P\) is a Markov matrix and \(P(x, y) > 0\) for all \(x, y\), then

    +
    +\[ + \| \psi P - \phi P \| < \| \psi - \phi \| + \text{ for all } \psi, \phi \in \dD + \text{ with } \psi \not= \phi +\]
    +
    +

    The proof follows from the strict triangle inequality, as opposed to the weak +triangle inequality we used to obtain (60).

    +

    See, for example, Proposition 3.1.2 of [LM94] or Lemma 8.2.3 of [Sta09].

    +
    +
    +

    Uniqueness

    +

    Irreducibility of a given Markov chain implies that there are no disjoint +absorbing sets.

    +

    This in turn leads to uniqueness of stationary distributions:

    +
    +

    Theorem 27

    +
    +

    Let \((P_t)\) be a UC Markov semigroup on \(S\). If \((P_t)\) is irreducible, then +\((P_t)\) has at most one stationary distribution.

    +
    +
    +

    Proof. Suppose to the contrary that \(\psi\) and \(\phi\) are both stationary for +\((P_t)\).

    +

    Since \((P_t)\) is irreducible, we know that \(P_1(x,y) > 0\) for all \(x,y \in S\).

    +

    If \(\psi \not= \phi\), then, due to positivity of \(P_1\), the strict inequality +in Lemma 26 holds.

    +

    At the same time, by stationarity, \(\| \psi P - \phi P \| = \| \psi - \phi +\|\). Contradiction.

    +
    +
    +

    Example 28

    +
    +

    An M/M/1 queue with parameters \(\mu, \lambda\) is a continuous time Markov chain \((X_t)\) on \(S = \ZZ_+\) with with intensity matrix

    +
    +(61)\[\begin{split} +Q=\begin{pmatrix} + -\lambda & \lambda & 0 & 0 & \cdots \\ + \mu & -(\mu + \lambda) & \lambda & 0 & \cdots \\ + 0 & \mu & -(\mu + \lambda) & \lambda & \cdots\\ + \vdots & \vdots & \vdots & \vdots & \ddots +\end{pmatrix} +\end{split}\]
    +

    The chain \((X_t)\) records the length of the queue at each moment in time.

    +

    The intensity matrix captures the idea that customers flow into the queue at +rate \(\lambda\) and are served (and hence leave the queue) at rate \(\mu\).

    +

    If \(\lambda\) and \(\mu\) are both positive, then there is a \(Q\)-positive +probability flow between any two states, in both directions, so the +corresponding semigroup \((P_t)\) is irreducible.

    +

    Theorem 27 now tells us that \((P_t)\) has at most one stationary +distribution.

    +
    +
    +
    +

    Stability from the Skeleton

    +

    Recall the definition of asymptotic stability given in (59).

    +

    Analogously, we call an individual Markov operator \(P\) asymptotically stable if +\(P\) has a unique stationary distribution \(\psi^*\) in \(\dD\) and +\(\psi P^n \to \psi^*\) as \(n \to \infty\) for all \(\psi \in \dD\).

    +

    The next result gives a connection between discrete and continuous stability.

    +

    The critical ingredient linking these two concepts is the contractivity in +(60).

    +
    +

    Lemma 29

    +
    +

    Let \((P_t)\) be a Markov semigroup. If there exists an \(s > 0\) such that the +Markov matrix \(P_s\) is asymptotically stable, then \((P_t)\) is asymptotically +stable with the same stationary distribution.

    +
    +
    +

    Proof. Let \((P_t)\) and \(s\) be as in the statement of Lemma 29.

    +

    Let \(\psi^*\) be the stationary distribution of \(P_s\). Fix \(\psi \in \dD\) and +\(\epsilon > 0\).

    +

    By stability of \(P_s\), we can take an \(n \in \NN\) such that +\(\| \psi P_s^n - \psi^* \| < \epsilon\).

    +

    Pick any \(t > sn\). Set \(h := t - sn\).

    +

    By the contractivity in (60) and \(P_{sn} = P_s^n\), we have

    +
    +\[ + \| \psi P_t - \psi^* \| + = \| \psi P_{sn} P_h - \psi^* P_h \| + \leq \| \psi P_{sn} - \psi^* \| + < \epsilon. +\]
    +

    Hence asymptotic stability holds

    +
    +
    +
    +

    Stability via Drift

    +

    In this section we address drift conditions, which are a powerful method for +obtaining asymptotic stability when the state space can be infinite.

    +

    The idea is to show that the state tends to drift back to a finite set over +time.

    +

    Such drift, when combined with the contractivity in +Lemma 26, is enough to give global stability.

    +

    The next theorem gives a useful version of this class of results.

    +
    +

    Theorem 30

    +
    +

    Let \((P_t)\) be a UC Markov semigroup with intensity matrix \(Q\). If \((P_t)\) is +irreducible and there exists a function \(v \colon S \to \RR_+\), a finite set +\(F \subset S\) and positive constants \(\epsilon\) and \(M\) such that

    +
    +\[\begin{split} + \sum_y Q(x, y) v(y) + \leq + \begin{cases} + M & \text{ if } x \in F \text{ and } + \\ + - \epsilon & \text{ otherwise } + \end{cases} +\end{split}\]
    +

    then \((P_t)\) is asymptotically stable.

    +
    +

    The proof of Theorem 30 can be found in [PichorRTKaminska12].

    +
    +

    Example 31

    +
    +

    Consider again the M/M/1 queue on \(\ZZ_+\) with intensity matrix (61).

    +

    Suppose that \(\lambda < \mu\).

    +

    It is intuitive that, in this case, the queue length will not +tend to infinity (since the service rate is higher than the arrival rate).

    +

    This intuition can be confirmed via Theorem 30, after setting \(v(i) = +i\).

    +

    Indeed, we have, for any \(i \geq 1\),

    +
    +\[ + \sum_{j \geq 0} Q(i, j) v(j) + = (i-1) \mu - i (\mu + \lambda) + (i+1) \lambda + = \lambda - \mu +\]
    +

    Setting \(F=\{0\}\) and \(\epsilon = \mu - \lambda\), we see that the conditions +of Theorem 30 hold.

    +

    Hence the associated semigroup \((P_t)\) is asymptotically stable.

    +
    +
    +

    Corollary 32

    +
    +

    If \((P_t)\) is an irreducible UC Markov semigroup and \(S\) is finite, then +\((P_t)\) is asymptotically stable.

    +
    +

    A solved exercise below asks you to confirm this.

    +
    +
    +
    +

    Exercises

    +
    +

    Exercise 1

    +

    Let \((P_t)\) be a Markov semigroup. True or false: +for this semigroup, every state \(x\) is accessible from itself.

    +
    +
    +

    Exercise 2

    +

    Let \((\lambda_k)\) be a bounded sequence in \((0, \infty)\).

    +

    A pure birth process starting at zero is a continuous time Markov process +\((X_t)\) on state space \(\ZZ_+\) with intensity matrix

    +
    +\[\begin{split} +Q=\begin{pmatrix} + -\lambda_0 & \lambda_0 & 0 & 0 & \cdots \\ + 0 & -\lambda_1 & \lambda_1 & 0 & \cdots \\ + 0 & 0 & -\lambda_2 & \lambda_2 & \cdots\\ + \vdots & \vdots & \vdots & \vdots & \ddots +\end{pmatrix} +\end{split}\]
    +

    Show that \((P_t)\), the corresponding Markov semigroup, has no stationary +distribution.

    +
    +
    +

    Exercise 3

    +

    Confirm that Theorem 30 implies Corollary 32.

    +
    +
    +
    +

    Solutions

    +
    +

    Solution to Exercise 1

    +

    The statement is true. With \(t=0\) we have \(P_t(x,x) = I(x,x) = 1 > 0\).

    +
    +
    +

    Solution to Exercise 2

    +

    Suppose to the contrary that \(\phi \in \dD\) and \(\phi Q = 0\).

    +

    Then

    +
    +\[ + (\phi Q)(j) + = \sum_{i \geq 0} \phi(i) Q(i, j) + = - \lambda_j \phi(j) + \lambda_j \phi(j+1) + = 0 +\]
    +

    It follows that \(\phi\) is constant on \(\ZZ_+\).

    +

    But \(\dD\) contains no constant functions when the state space is infinite. +(Why?)

    +

    Contradiction.

    +
    +
    +

    Solution to Exercise 3

    +

    Let \((P_t)\) be an irreducible UC Markov semigroup and let \(S\) be finite.

    +

    Pick any positive constants \(M, \epsilon\) and set \(v = M\) and \(F = S\).

    +

    We then have

    +
    +\[ + \sum_y Q(x, y) v(y) + = M \sum_y Q(x, y) + = 0 +\]
    +

    Hence the drift condition in Theorem 30 holds and \((P_t)\) is +asymptotically stable.

    +
    @@ -428,7 +907,7 @@

    Asymptotic Stabiltiy - A Probabilistic View + UC Markov Semigroups Bibliography

    diff --git a/generators.html b/generators.html index 480785a..f56fae6 100644 --- a/generators.html +++ b/generators.html @@ -113,11 +113,6 @@

    Continuous Time Markov Chains

    UC Markov Semigroups -
  • - - A Probabilistic View - -
  • Stationarity and Ergodicity @@ -721,7 +716,7 @@

    Solution to Exercise 3
    2

    An excellent introduction to operator semigroups, combined with -applications to PDEs and Markov processes, can be found in [applebaum2019semigroups].

    +applications to PDEs and Markov processes, can be found in [App19].

    3

    Convergence of the sum in (40) follows from boundedness of \(A\) and the fact that \(\linop\) is a Banach space.

    diff --git a/genindex.html b/genindex.html index 28743b0..f5893e6 100644 --- a/genindex.html +++ b/genindex.html @@ -112,11 +112,6 @@

    Continuous Time Markov Chains

    UC Markov Semigroups

    • -
  • - - A Probabilistic View - -
  • Stationarity and Ergodicity diff --git a/intro.html b/intro.html index bf514f9..6f1f3f3 100644 --- a/intro.html +++ b/intro.html @@ -112,11 +112,6 @@

    Continuous Time Markov Chains

    UC Markov Semigroups
    • -
  • - - A Probabilistic View - -
  • Stationarity and Ergodicity @@ -207,8 +202,43 @@

    Continuous Time Markov Chains

    Continuous Time Markov Chains

    -

    Introductory text to be added.

    -

    Uses — list applications.

    +

    Authors: Thomas J. Sargent and John +Stachurski

    +

    This lecture series provides a short introduction to the +fascinating field of continuous time Markov chains. It will, in time, be +integrated into our QuantEcon lectures. +Focus is shared between theory, applications and computation. Mathematical +ideas are combined with computer code to help clarify and build intuition, as +well as to bridge the gap between theory and applications. +The presentation is relatively rigorous but the aim is towards applications +rather than mathematical curiosities (which are plentiful, if one starts to +look). Applications are mainly drawn from economics and operations research.

    +
    +

    Solved exercises

    +

    There are many solved exercises and we recommend readers attempt all of them, +or at least review the solutions.

    +
    +
    +

    Computer code

    +

    The code is written in Python and is accelerated through a combination of +NumPy (vectorized code) and just-in-time compilation +(via Numba). +QuantEcon provides a fast-paced introduction to scientific computing with Python that covers these topics.

    +
    +
    +

    Background: Markov chains in discrete time

    +

    The lectures are well suited to those who have some knowledge of discrete time +Markov chains and wish to learn more about their continuous time cousins. A +suitable preliminary discussion of discrete time Markov chains can be found +here.

    +
    +
    +

    Prerequisites: Probability and Analysis

    +

    Readers are assumed to be familiar with probability and a small amount of +analysis. Later lectures, which deal with infinite state spaces, assume that +require that readers are comfortable with the basics of linear analysis in Banach space.

    +
    +

    The lectures are written using Jupyter Book.

    diff --git a/kolmogorov_bwd.html b/kolmogorov_bwd.html index 32a9a5a..7fb6a84 100644 --- a/kolmogorov_bwd.html +++ b/kolmogorov_bwd.html @@ -113,11 +113,6 @@

    Continuous Time Markov Chains

    UC Markov Semigroups
    • -
  • - - A Probabilistic View - -
  • Stationarity and Ergodicity @@ -234,8 +229,8 @@

    Continuous Time Markov Chains
  • - - Embedded Jump Chain Algorithm + + Jump Chain Algorithm
    • @@ -402,8 +397,8 @@

    Motivation -

    Embedded Jump Chain Algorithm

    +
    +

    Jump Chain Algorithm

    We start with three primitives

    1. An initial condition \(\psi\),

      2. @@ -420,7 +415,7 @@

      Motivation\(y\) as the new state for the process and repeat.

      Here is the same algorithm written more explicitly:
      diff --git a/memoryless.html b/memoryless.html index 89960a0..8989b4e 100644 --- a/memoryless.html +++ b/memoryless.html @@ -113,11 +113,6 @@

      Continuous Time Markov Chains

      UC Markov Semigroups -
    2. - - A Probabilistic View - -
    3. Stationarity and Ergodicity diff --git a/objects.inv b/objects.inv index 0290660886fc619e4d5fba6ff0cfd6a6fec1f998..6534d56fdb1257b5cb1fd3d5b26f9fe942f9a447 100644 GIT binary patch delta 834 zcmV-I1HJr#2dW2(4a|sH7IH%F(x@GNqOnl zcPPo0WF)n6FklboJ3n^GI0^E7- zwR&^LMp2Bh#f@9Zi}CnFS{mUc8mcU%Y!j*l9=6`mD>t$iF*h3zJqYV7) zVIN`yHp4^;>30=AK@rfcwF0vT&#Sm>@Z%<7^MgZ_a;cUY^wIsG(e4zuCgR{V3SJBP zQd(e`vb%$ip-+@bwtcrG~d`A&sOTAzFx1g_~t6vlAr2Z4}(@ zEyg^SDx?^Uj(??Sna_bqvj*cSRHiWjbslA4z;d#nX!9240wm?I1ORl0EH{2%FoS#5}CgYISl*U`^ zCRiEj=&q57O^Kzeq-S@Ptn51HR766qsffN-aX<|z958k8N`?6*Uo!1i2!jTj%dP=DOPR|fUb&DiKFYGAHp$NHim0_P`g-Ts`N#wbWats>d-CF_SqD7 z==o$ieSc3$;*j1^K+X0stTrRL83K@52Hq2DEZW{WjqB8?o8@uosuY_hr9m&i(&lQX zs0Z!OR%c7~YVIOh2G2}!u^a6V8ayVYf@Yo#S}PqX`MHj8K!ydC0?RGE53j&Tk)tIs zFPulJne$Q%fuVE`O;?S^RHt&^oRfltbKoKiw12X#Z}o3==m6g|ZllUAl;US?kS1%( z;dqWnF;A*Jla$p{I3Q}5Jjo$5S3L`C1_e+lXo9O5yfGfCSCNu&z^%ypRJCot(5v%% zolOUvpV-**Zc8lpS5VimCHkytqg8LEl=OvF?KIK_GMQxRuVJI Mi+mUT1AMVxE=fF^r2qf` delta 793 zcmV+!1Lpjy2Z0BWb$?acP8%^0efL*bg2$!;s$S?Dw^UFep+fr_dDpYMCbnm5dkN{+ zckI0*+JuKloAEj0nKKt>m^s||wK_RE`dBnJ4+^c&+z0tpiwrbh9P>X83UFX^@b9Z& z^uZxtupF&_Y)l=TupW$>Q;cwHd|K5G>z?uufhzjv_k=GUHh*>U67+gbCRVn3Qn2Z% z7ZK%}6dVIrqExvG+r*NXN6u(XBz_-6O2tK-u##U509FX4R|d?BkinqaYVci>5w(b@ zC4DpwI2QbNz*zdRLQz~IfK)zr5WwU+Rk-Y+{6QIb$Ct_~`V*rCR%qO;3RUiK5@Cx7 zyCa=MrgTg(8h<@6(D6_KSM~{PpisF%3Hm)N!GaTH@l(-8YbRP?7oQ~iHXKL3vJ#%> zq4n05eH}jui6$UC*Z)`dkS`q?x&jRZr*a;UUp?1oDQ+b+t8gQnJUKf43>pj>{T@B3 zY;ZVDFd5qDsE2~G4K_YW&u^2g{Gt+6L}ETs5ksru9)C5Ykh3QB@^U8p^&tFh6rK%* zSjnvwW{2wmy=c*At(Vw&qvOkXq;k(5CH3F#v7@MNkc60p&En|hF?XR$oI6>gHdvw~ zN7+X9ey&UgQi9|+#B62bXT+res|=fSrX_gA8~;Ao__@f}vC{nNcatV|>Zy zwj2xMMTJN$aGVy1Y+s%|()IFD?K4bi-}6hTfr>$+nP%$R21qX?so+7lyqddeZT+TB z=Xat6My-IDuPNLQZw$A|S0-QiaqCYce|gs4`}3l2-tnzq8=-wg*PpB{ZlaF6Z=M*Z XmlD3j@^h7Gg{2d9wPk++RS+WRr~QHp diff --git a/poisson.html b/poisson.html index 8ac7e26..4297a45 100644 --- a/poisson.html +++ b/poisson.html @@ -113,11 +113,6 @@

      Continuous Time Markov Chains

      UC Markov Semigroups
      4. -
    4. - - A Probabilistic View - -
    5. Stationarity and Ergodicity diff --git a/prob_view.html b/prob_view.html deleted file mode 100644 index e02f373..0000000 --- a/prob_view.html +++ /dev/null @@ -1,512 +0,0 @@ - - - - - - - - A Probabilistic View — Continuous Time Markov Chains - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
      -
      - - - - - - - - -
      - -
      -
      -
      -
      - - - -
      - - - -
      - - - - - - -
      - -
      - - - - - - - - -
      - -
      - - - - - - -
      -
      - -
      - - -
      -
      - Contents -
      - -
      -
      -
      -
      - -
      - -
      -

      A Probabilistic View

      -
      -

      Overview

      -

      Let \(S\) be a countable state space and let \(Q\) be a conservative intensity -matrix on \(S\).

      -

      We know that \(Q\) generates a UC Markov semigroup, and from this semigroup we -can produce a continuous time Markov chain via the associated joint distribution.

      -

      This method of building chains from intensity matrices is important from a -theoretical perspective but less helpful in terms of intuition and simulation.

      -

      In this lecture we provide another point of view.

      -

      In particular, we provide two different ways to build a Markov chain with -intensity matrix \(Q\).

      -

      One is via jump chains with state dependent jump rates.

      -

      The second is another probabilistic construction, sometimes called Gillespie’s -Algorithm.

      -

      These two constructions provide intuition from multiple perspectives and -valuable simulation algorithms.

      -

      We will use the following imports

      -
      -
      -
      import numpy as np
-import scipy as sp
-import matplotlib.pyplot as plt
-import quantecon as qe
-from numba import njit
-from scipy.linalg import expm
-
      -
      -
      -
      -
      -
      -

      From Intensity Matrix to Jump Chain

      -

      Let us agree to call \((\lambda, K)\) a jump chain pair if \(\lambda\) is a -map from \(S\) to \(\RR_+\) and \(K\) is a Markov matrix on \(S\).

      -

      It is easy to verify that the matrix \(Q\) on \(S\) defined by

      -
      -(53)\[ - Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) -\]
      -

      is an intensity matrix.

      -

      (We saw in an earlier lecture that \(Q\) is the intensity -matrix for the jump chain \((X_t)\) built via Algorithm 6 from jump -chain pair \((\lambda, K)\).)

      -

      As we now show, every intensity matrix admits the decomposition in -(53) for some jump chain pair.

      -
      -

      Construction

      -

      Given a intensity matrix \(Q\), set

      -
      -\[ - \lambda(x) := -Q(x, x) - \qquad (x \in S) -\]
      -

      Next we build \(K\), first along the principle diagonal via

      -
      -\[\begin{split} - K(x,x) = - \begin{cases} - 0 & \text{ if } \lambda(x) > 0 - \\ - 1 & \text{ otherwise} - \end{cases} -\end{split}\]
      -

      Thus, if the rate of leaving \(x\) is positive, we set \(K(x,x) = 0\), so that the -embedded jump chain moves away from \(x\) with probability one when the next -jump occurs.

      -

      Otherwise, when \(Q(x,x) = 0\), we stay at \(x\) forever, so \(x\) is an absorbing -state.

      -

      Off the principle diagonal, where \(x \not= y\), we set

      -
      -\[\begin{split} - K(x,y) = - \begin{cases} - \frac{Q(x,y)}{\lambda(x)} & \text{ if } \lambda(x) > 0 - \\ - 0 & \text{ otherwise } - \end{cases} -\end{split}\]
      -

      The exercises below ask you to confirm that, for \(\lambda\) and \(K\) just defined,

      -
        -

      1. \((\lambda, K)\) is a jump chain pair and

        2. -

      2. the intensity matrix \(Q\) satisfies (53).

        3. -
      -

      We call \((\lambda, K)\) the jump chain decomposition of \(Q\).

      -

      We summarize in a lemma.

      -
      -

      Lemma 21

      -
      -

      A matrix \(Q\) on \(S\) is an intensity matrix if and only if there exists a jump -chain pair \((\lambda, K)\) such that (53) holds.

      -
      -
      -
      -

      The Conservative Case

      -

      We know from Example 20 that an intensity matrix \(Q\) is conservative -if and only if \(\lambda\) is bounded.

      -

      Moreover, we saw in Theorem 17 that the pairing between conservative -intensity matrices and UC Markov semigroups is one-to-one.

      -

      This leads to the following result.

      -
      -

      Theorem 22

      -
      -

      On \(S\), there exists a one-to-one correspondence between the following sets of -objects:

      -
        -

      1. The set of all jump chain pairs \((\lambda, K)\) such that \(\lambda\) is bounded.

        2. -

      2. The set of all conservative intensity matrices.

        3. -

      3. The set of all UC Markov semigroups.

        4. -
      -
      -
      -
      -

      Simulation

      -

      In view of the preceding discussion, we have a simple way to simulate a Markov -chain given any conservative intensity matrix \(Q\).

      -

      The steps are

      -
        -

      1. Decompose \(Q\) into a jump chain pair \((\lambda, K)\).

        2. -

      2. Simulate via Algorithm 6.

        3. -
      -

      Recalling our discussion of the Kolmogorov backward equation, we know that -this produces a Markov chain with Markov semigroup -\((P_t)\) where \(P_t = e^{tQ}\) for \(Q\) satisfying (53).

      -

      (Although our argument assumed finite \(S\), the proof goes through when -\(S\) is contably infinite and \(Q\) is conservative with very minor changes.)

      -

      In particular, \((X_t)\) is a continuous time Markov chain with intensity matrix -\(Q\).

      -
      -
      -
      -

      The Gillespie Algorithm

      -

      Exponential clocks.

      -

      Show by simulation that distributions coincide with \(\psi_0 P_t\).

      -
      -
      -

      Exercises

      -
      -

      Exercise 1

      -

      Let \(Q\) be any intensity matrix on \(S\).

      -

      Prove that the jump chain decomposition of \(Q\) is in fact a jump chain pair.

      -

      Prove that, in addition, this decomposition \((\lambda, K)\) satisfies (53).

      -
      -
      -
      -

      Solutions

      -
      -

      Solution to Exercise 1

      -

      Let \(Q\) be an intensity matrix and let \((\lambda, K)\) be the jump chain -decomposition of \(Q\).

      -

      Nonnegativity of \(\lambda\) is immediate from the definition of an intensity -matrix.

      -

      To see that \(K\) is a Markov matrix we fix \(x \in S\) and suppose first that -\(\lambda(x) > 0\).

      -

      Then

      -
      -\[ - \sum_y K(x, y) - = \sum_{y \not= x} K(x,y) - = \sum_{y \not= x} \frac{Q(x,y)}{\lambda(x)} - = 1 -\]
      -

      If, on the other hand, \(\lambda(x) = 0\), then -\(\sum_y K(x, y) = 1\), is immediate from the definition.

      -

      As \(K\) is nonnegative, we see that \(K\) is a Markov matrix.

      -

      Thus, \((\lambda, K)\) is a valid jump chain pair.

      -

      The proof that \(Q\) and \((\lambda, K)\) satisfy (53) is mechanical and -the details are omitted.

      -

      (Try working case-by-case, with \(\lambda(x) = 0, x=y\), \(\lambda(x) > 0, x=y\), etc.)

      -
      -
      -
      - - - - -
      - -
      -
      - - -
      - - UC Markov Semigroups - Stationarity and Ergodicity - -
      -
      -
      -

      - - By John Stachurski
      - - © Copyright 2020.
      -

      -
      -
      -
      - - -
      -
      - - - - - \ No newline at end of file diff --git a/proof-proof.html b/proof-proof.html index 2759890..d7a3c05 100644 --- a/proof-proof.html +++ b/proof-proof.html @@ -118,11 +118,6 @@

      Continuous Time Markov Chains

      UC Markov Semigroups
      6. -
    6. - - A Probabilistic View - -
    7. Stationarity and Ergodicity @@ -208,6 +203,8 @@

      Proof Index

      erlexp | example-0 | example-1 | + example-5 | + example-8 | exp_unique | imatjc | intvsmk | @@ -215,12 +212,19 @@

      Proof Index

      jccs | jctosg | lemma-1 | + perimposs | scintcon | + sdrift | + sfinite | + stabskel | statfromq | + strictcontract | theorem-0 | theorem-1 | theorem-2 | + theorem-5 | ucsgec | + uniirr | usmg
      8. @@ -264,7 +268,7 @@

      Proof Index

      equivirr (ergodicity) - lemma + theorem   erlexp @@ -290,6 +294,22 @@

      Proof Index

      example-1 (uc_mc_semigroups) example   + + example-5 + + + + example-5 (ergodicity) + example +   + + example-8 + + + + example-8 (ergodicity) + example +   exp_unique @@ -303,7 +323,7 @@

      Proof Index

      - imatjc (prob_view) + imatjc (uc_mc_semigroups) lemma   @@ -346,6 +366,14 @@

      Proof Index

      lemma-1 (kolmogorov_bwd) lemma   + + perimposs + + + + perimposs (ergodicity) + corollary +   scintcon @@ -354,6 +382,30 @@

      Proof Index

      scintcon (uc_mc_semigroups) lemma   + + sdrift + + + + sdrift (ergodicity) + theorem +   + + sfinite + + + + sfinite (ergodicity) + corollary +   + + stabskel + + + + stabskel (ergodicity) + lemma +   statfromq @@ -362,6 +414,14 @@

      Proof Index

      statfromq (ergodicity) theorem   + + strictcontract + + + + strictcontract (ergodicity) + lemma +   theorem-0 @@ -383,7 +443,15 @@

      Proof Index

      - theorem-2 (ergodicity) + theorem-2 (kolmogorov_bwd) + theorem +   + + theorem-5 + + + + theorem-5 (uc_mc_semigroups) theorem   @@ -394,6 +462,14 @@

      Proof Index

      ucsgec (generators) theorem   + + uniirr + + + + uniirr (ergodicity) + theorem +   usmg diff --git a/reports/ergodicity.log b/reports/ergodicity.log new file mode 100644 index 0000000..8ac21c1 --- /dev/null +++ b/reports/ergodicity.log @@ -0,0 +1,106 @@ +Traceback (most recent call last): + File "/home/john/anaconda3/lib/python3.7/site-packages/jupyter_cache/executors/utils.py", line 56, in single_nb_execution + record_timing=False, + File "/home/john/anaconda3/lib/python3.7/site-packages/nbclient/client.py", line 968, in execute + return NotebookClient(nb=nb, resources=resources, km=km, **kwargs).execute() + File "/home/john/anaconda3/lib/python3.7/site-packages/nbclient/util.py", line 72, in wrapped + return just_run(coro(*args, **kwargs)) + File "/home/john/anaconda3/lib/python3.7/site-packages/nbclient/util.py", line 51, in just_run + return loop.run_until_complete(coro) + File "/home/john/anaconda3/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete + return future.result() + File "/home/john/anaconda3/lib/python3.7/site-packages/nbclient/client.py", line 509, in async_execute + cell, index, execution_count=self.code_cells_executed + 1 + File "/home/john/anaconda3/lib/python3.7/site-packages/nbclient/client.py", line 747, in async_execute_cell + self._check_raise_for_error(cell, exec_reply) + File "/home/john/anaconda3/lib/python3.7/site-packages/nbclient/client.py", line 671, in _check_raise_for_error + raise CellExecutionError.from_cell_and_msg(cell, exec_reply['content']) +nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell: +------------------ +def unit_simplex(angle): + + fig = plt.figure(figsize=(8, 6)) + ax = fig.add_subplot(111, projection='3d') + + vtx = [[0, 0, 1], + [0, 1, 0], + [1, 0, 0]] + + tri = Poly3DCollection([vtx], color='darkblue', alpha=0.3) + tri.set_facecolor([0.5, 0.5, 1]) + ax.add_collection3d(tri) + + ax.set(xlim=(0, 1), ylim=(0, 1), zlim=(0, 1), + xticks=(1,), yticks=(1,), zticks=(1,)) + + ax.set_xticklabels(['$(1, 0, 0)$'], fontsize=12) + ax.set_yticklabels(['$(0, 1, 0)$'], fontsize=12) + ax.set_zticklabels(['$(0, 0, 1)$'], fontsize=12) + + ax.xaxis.majorTicks[0].set_pad(15) + ax.yaxis.majorTicks[0].set_pad(15) + ax.zaxis.majorTicks[0].set_pad(35) + + ax.view_init(30, angle) + + # Move axis to origin + ax.xaxis._axinfo['juggled'] = (0, 0, 0) + ax.yaxis._axinfo['juggled'] = (1, 1, 1) + ax.zaxis._axinfo['juggled'] = (2, 2, 0) + + ax.grid(False) + + return ax + +Q = np.array(Q) +ψ_00 = np.array((0.01, 0.01, 0.99)) +ψ_01 = np.array((0.01, 0.99, 0.01)) +ψ_02 = np.array((0.99, 0.01, 0.01)) + +ax = unit_simplex(angle=50) + +def flow_plot(ψ, h=0.001, n=400, angle=50): + colors = cm.jet_r(np.linspace(0.0, 1, n)) + + x_vals, y_vals, z_vals = [], [], [] + for t in range(n): + x_vals.append(ψ[0]) + y_vals.append(ψ[1]) + z_vals.append(ψ[2]) + ψ = ψ @ expm(h * Q) + + ax.scatter(x_vals, y_vals, z_vals, c=colors, s=20, alpha=0.2, depthshade=False) + +flow_plot(ψ_00) +flow_plot(ψ_01) +flow_plot(ψ_02) + +# Add stationary distribution +#P_1 = expm(Q) + +#mc = qe.MarkovChain(P_1) +#ψ_star = mc.stationary_distributions +#ax.scatter(0.2, 0.2, 0.6, c='k', s=20, alpha=0.2, depthshade=False) + +plt.show() +------------------ + +--------------------------------------------------------------------------- +NameError Traceback (most recent call last) + in  + 53 ax.scatter(x_vals, y_vals, z_vals, c=colors, s=20, alpha=0.2, depthshade=False) + 54  +---> 55 flow_plot(ψ_00) + 56 flow_plot(ψ_01) + 57 flow_plot(ψ_02) + + in flow_plot(ψ, h, n, angle) + 42  + 43 def flow_plot(ψ, h=0.001, n=400, angle=50): +---> 44 colors = cm.jet_r(np.linspace(0.0, 1, n)) + 45  + 46 x_vals, y_vals, z_vals = [], [], [] + +NameError: name 'cm' is not defined +NameError: name 'cm' is not defined + diff --git a/search.html b/search.html index b906232..e34ae87 100644 --- a/search.html +++ b/search.html @@ -116,11 +116,6 @@

      Continuous Time Markov Chains

      UC Markov Semigroups -
    8. - - A Probabilistic View - -
    9. Stationarity and Ergodicity diff --git a/searchindex.js b/searchindex.js index b25b7a8..65fb89e 100644 --- a/searchindex.js +++ b/searchindex.js @@ -1 +1 @@ -Search.setIndex({docnames:["ergodicity","generators","intro","kolmogorov_bwd","kolmogorov_fwd","markov_prop","memoryless","poisson","prob_view","uc_mc_semigroups","zreferences"],envversion:{"sphinx.domains.c":1,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":1,"sphinx.domains.index":1,"sphinx.domains.javascript":1,"sphinx.domains.math":2,"sphinx.domains.python":1,"sphinx.domains.rst":1,"sphinx.domains.std":1,"sphinx.ext.intersphinx":1,sphinx:56},filenames:["ergodicity.md","generators.md","intro.md","kolmogorov_bwd.md","kolmogorov_fwd.md","markov_prop.md","memoryless.md","poisson.md","prob_view.md","uc_mc_semigroups.md","zreferences.md"],objects:{},objnames:{},objtypes:{},terms:{"100_000":3,"10_000":[6,7],"1_000":6,"1e5":7,"\u03b1":[3,5],"\u03b3":3,"\u03b8":7,"\u03b8_val":7,"\u03bb":[3,5,6,7],"\u03c8":[3,4,5],"\u03c8_0":[3,5],"\u03c8_00":4,"\u03c8_01":4,"\u03c8_02":4,"\u03c8_t":3,"abstract":[1,4,5],"break":5,"case":[0,1,3,6,7],"class":[6,9],"final":[3,5],"function":[3,4,5,6,7,9,10],"import":[0,1,3,4,5,6,7,8],"long":[5,6],"new":[3,5,7],"return":[1,3,4,5,6,7],"switch":5,"true":[0,1,3,4,5,7],"try":[3,4,5,7,8,9],"while":[1,3,4,5,6,7,9],And:4,Are:6,But:[4,6,9],Doing:[],For:[0,1,3,4,5,6,7,9],Its:5,M_s:7,N_s:[5,7],Not:5,ODE:[0,1,4],ODEs:[1,3],One:[1,3,5,7,8],P_s:[3,5],Such:1,That:[5,7],The:0,Then:[5,6,7,8],There:[0,1,4,5],These:[4,5,8],U_s:1,Use:0,Uses:2,Using:[1,4,6,7],V_s:3,With:5,X_s:5,__call__:6,__init__:6,_axinfo:4,a_n:6,abbrevi:[1,5],abl:1,about:[1,3,5,6],abov:[1,3,4,5,6,7,9],absorb:8,access:0,accord:[4,5],across:6,actual:3,adam:10,adapt:4,add:5,add_collection3d:4,add_subplot:[4,5],added:[0,2],addit:[1,5,7,8,9],admit:8,adopt:[1,3],affirm:5,after:[1,3,4,5,6],again:[1,4,5,6,7,9],agre:[5,7,8],aim:[1,5,9],alcohol:6,algebra:[1,5],algorithm:[4,6,10],align:[3,5],all:[0,1,3,4,5,6,7,8,9],allow:[1,3,5],along:[3,4,5,8,9],alpha:[1,3,4,5,6,7],alreadi:[4,5,6,7],also:[1,3,4,5,6,7,9],altern:[0,5,7,9],although:[4,5,6,8],alwai:[1,3,4,5,9],american:10,amount:[3,5,6],analog:[0,1,3,4,5,9],analysi:[6,10],analyt:[3,4],analyz:5,angl:4,ani:[0,1,3,4,5,6,7,8,9],anoth:[1,3,5,6,7,8,9],answer:[3,5,7],aperiod:0,appeal:[7,9],appear:[],append:[4,5],applebaum2019semigroup:1,appli:[3,4,5,7,9],appliat:9,applic:[1,2,4,5,9,10],approach:[3,4,5,10],appropri:1,approx:[6,7],approxim:[3,7],arang:[3,5,7],arbitrari:[1,3,5,7,9],argu:4,argument:[1,3,4,5,8,9],aris:[1,9],arrai:[4,5],arriv:[3,5,6,7],art3d:[0,4],ask:[3,5,7,8,9],assert:4,associ:[0,1,3,4,7,8,9],assum:[0,1,3,4,5,8,9],assumpt:[3,4],attent:9,attest:6,author:[],autom:[],automat:5,autonom:4,avoid:[1,4],awai:8,axes3d:[0,4,5],axes:7,axi:4,b_n:6,back:[4,5],backward:[1,8,9],banach:[1,9],bar:[0,3,5],basic:1,becaus:[0,1,3,4,5,6,7,9],becom:[3,5,6,7],been:[4,5],befor:[4,5],begin:[3,4,5,6,8,9],behav:[],behind:7,being:[1,4],below:[1,3,4,5,6,7,8,9],benefici:[],bernoulli:5,bet:6,beta:1,better:1,between:[1,3,4,5,6,7,8,9],beverag:6,big:1,binari:[5,7],binom:[3,5,6,7],binom_v:7,binomi:[3,5,7],black:6,bob05:[1,10],bobrowski:10,book:5,both:[4,5,6,9],bound:[1,5,8],bounded:[1,9],brief:9,brownian:10,build:[5,8],built:8,busi:10,c_0:[1,9],calcul:[3,5],calculu:10,call:[0,1,3,4,5,6,7,8,9],cambridg:10,can:[0,1,3,4,5,6,8,9],cannot:[1,5],care:[1,4],carri:[4,5],cartiesian:5,casino:6,cauchi:1,cdf:[6,7],cdot:[0,1,3,4,5,7,9],center:7,certain:4,certainli:1,chain:[1,6,7,9,10],challeng:1,chang:[1,3,4,5,6,8],chapman:5,chapter:1,character:0,charcter:9,check:[1,4,5,9],choic:9,choos:9,chosen:[4,5],circumv:1,claim:[1,3,6,7,9],clarifi:[1,3,4,5,6],clear:[3,7],clearer:5,clearli:9,clock:8,close:[5,6],code:[3,4],coeffici:3,coincid:[1,8],collect:[5,9],colon:[3,9],color:[4,5],column:4,combin:[1,3,4,9],come:[1,6],common:[5,6],commonli:4,compar:[6,7],comparison:4,complet:[1,5,6,9],complex:3,complic:1,composit:[1,9],comput:[5,10],concern:1,conclud:[4,5,7,9],concret:5,condit:[0,3,4,5],confirm:[3,4,6,8,9],confus:5,connect:[1,4,5,6,7,9],consequ:7,conserv:[0,9],consid:[0,1,4,5,6,9],consist:[3,5,6,9],constant:[1,3,4,9],construct:[3,4,6,7],consum:7,contabl:8,contain:9,context:4,continu:[3,6,7,8,9,10],contradict:9,conveni:[3,5],convent:5,converg:[1,6,7,9],convergence_plot:4,convers:[0,4,6,9],convert:3,convinc:3,cool:[4,5],corollari:5,correct:[4,5,7],correspond:[4,5,8,9],could:5,countabl:[0,4,8,9],cours:5,cover:[5,9],crc:10,creat:[4,5],cross:[5,6],cumsum:7,current:[4,5,6,9],curv:[],custom:[3,5,6,7],d_t:0,daniel:10,darkblu:4,data:5,date:[4,5],decompos:[8,9],decomposit:8,decreas:6,deduc:9,deep:7,def:[3,4,5,6,7],defer:4,defin:[1,3,4,5,6,7,8,9],definit:[0,1,3,4,5,6,7,8,9],delai:5,delet:5,denot:[1,3,4,5,6],densiti:6,depend:[4,5,6,7,8],depthshad:4,deriv:[1,3,4,9],describ:[3,4,5],descript:[1,3,4,5],desir:5,despit:1,detail:[1,5,7,8,9],determin:[5,9],determinist:3,develop:5,diagon:[4,8],differ:[0,3,5,6,7,8],differenti:[4,9],diffexpmap:[],difficult:[3,5,7],diffus:[1,4],dimension:[1,3,4,5,9],direct:[1,5],directli:[1,4,9],discourag:6,discov:[],discret:[1,6,7,9],discuss:[1,3,4,5,6,7,8,9],displai:[0,5],distinct:0,distract:4,distribut:[1,3,7,8,9],doe:[1,5],domain:1,domin:9,done:[1,3],dougla:10,down:[3,4,5],downarrow:[1,4],downsid:[],draw:[3,5,6,7],draw_nt:7,draw_nt_sampl:7,draw_x:[3,6],drawn:3,driven:1,dtype:3,due:[3,5,6,7],dynam:4,each:[1,3,4,5,6,7,9],earli:[4,5],earlier:[3,7,8,9],easi:[0,1,3,4,5,8],easier:[0,3,9],easili:[1,3,4,9],econom:5,economist:5,ecuc:[],effici:5,either:[5,6,7],elaps:6,eleg:5,element:[0,1,3,4,5,9,10],ell:5,ell_1:[0,1,9],els:[3,5,6],elsewher:5,embed:[8,9],emphasi:4,empir:[6,7],empirical_exceed:6,emploi:4,empti:[3,6,7],empty_lik:3,encourag:3,end:[0,3,4,5,6,7,8,9],enjoi:1,enough:[5,9],ensur:9,enter:6,entir:[1,3,5,6],entri:[3,6],environ:7,equal:[1,3,4,5,9],equat:[1,5,8,9,10],equival:[0,1,4,5,7,9],erlang:[6,7],essenc:[],essenti:9,establish:[0,1,9],etc:[7,8],etienn:10,evalu:[3,5,7],even:[1,5],evenli:6,event:6,everi:[0,1,4,5,8,9],everywher:[1,9],evolut:[1,4,5],evolv:[4,5],exact:[4,9],exactli:5,examin:[4,7],exampl:[1,3,4,6,7,8,9],exceed:6,excel:1,exercis:[],exist:[0,1,4,5,6,7,8,9],exp:[3,5,6,7],exp_uniqu:[],expect:[3,4,5,7,9],experi:[3,7],explicitli:[3,4,5],explod:[1,9],explor:[1,5],explos:9,expm:[3,4,5,8],exponenti:[4,5,8,9],exposit:5,express:[1,5,7,9],extend:4,extens:[1,5],fact:[1,3,4,5,6,7,8,9],factori:[6,7],fail:[1,3,5,6],fall:[5,6,9],fals:[4,5],famili:[5,9],familiar:[1,5],far:[5,6],featur:7,feel:6,ff_:5,field:[],fifth:6,fig:[3,4,5,6,7],figsiz:[4,5,7],figur:[0,3,4,5,6,7],financ:10,finer:6,finit:[0,1,3,4,7,8],firm:5,first:[0,1,3,4,5,6,7,8,9],fit:[1,6,7],five:6,fix:[1,3,4,5,6,7,8],flatten:7,flow:[0,1,4,9],flow_fig:5,flow_plot:4,fncoex:[],fnpde:[],focu:[1,3,9],focus:5,foguel:0,fokker:4,follow:[0,1,3,4,5,6,7,8,9],fontsiz:[4,7],foobar:[],foral:[1,6],forev:8,forget:6,form:[1,3,5,6,9],formal:5,formula:5,fortun:[5,9],forward:[1,5,9],found:[1,4,7],four:6,frac:[0,1,3,4,5,6,7,8,9],fraction:6,framework:1,fran:10,free:6,freeli:7,from:[0,1,5,7,9],full:[1,3,5],fulli:1,fundament:[4,5],further:[3,5,6,7],futur:[5,6],gall:10,gamma:3,gener:[0,3,4,5,7,8,9],genom:10,geometr:[3,5,7],geq:[0,1,3,4,5,6,7,9],get:[0,4,5,6],give:[1,3,4,5,9],given:[0,1,3,4,5,6,7,8,9],glue:5,goe:[5,8],good:[6,7],govern:5,great:3,grid:[4,5,6,7],grow:9,guess:3,h_n:1,halv:6,hand:[1,3,5,6,7,8,9],happen:[],hard:[1,3,9],harder:3,has:[0,1,3,4,5,6,7,9],hat:[3,4,5,7,9],have:[0,1,3,4,5,6,7,8,9],haven:4,help:[3,5,6,7,8],henc:[0,1,3,4,5,6,7,9],henceforth:[4,5],here:[0,3,4,5,6,7,9],high:1,highli:[],hill:9,hint:[1,4,5],histogram:[3,5],histori:[5,6],hline:7,hold:[0,1,3,4,5,6,8,9],homogen:[4,5],hope:9,hot:[4,5],hour:6,how17:[7,10],how:[1,4,5,6,9],howard:10,howev:[1,9],hypothesi:5,idea:[0,1,3,4,5,6,7],ideal:1,ident:[1,3,5,9],identif:9,identifi:[0,1,4,5,9],iff:3,iid:[3,5,6,7],illustr:[5,6,7],imagin:6,immedi:[3,4,5,8,9],impli:[0,1,3,5,9],implic:4,implicitli:3,impos:4,improv:7,includ:[0,5],increas:[6,7],increment:5,inde:[1,3,5,7],indepen:7,independ:[3,5,6],independent_draw:3,index:5,indic:[1,4,5,7],indirectli:[],indistinguish:6,induc:9,induct:[5,7],inequ:[0,9],infin:9,infinit:[1,4,5,8,9],infinitesim:[1,3,4],inflow:4,inform:[3,5,7],infti:[1,3,4,5,6,7,9],initi:[1,3,4,5,9],input:[3,5],insert:[5,7,9],insight:3,insist:5,instead:[3,7],instinct:6,int64:3,int_0:3,integ:[5,6,7],intens:[0,1,4,5],interest:[1,5,6],interpret:[1,5],interv:[5,6,7],introduc:[1,5,9],introduct:[1,5,6,10],introductori:2,intuit:[3,4,5,7,8,9],intvsmk:[],invari:4,inventori:4,invert:1,investig:[3,7],involv:5,isaac:3,issu:[1,9],iter:5,its:[0,1,5,6,7],itself:[1,4,5,9],j_0:[3,5,7],j_1:3,j_k:[3,5,7],j_n:[6,7],j_val:5,jame:10,jctosg:[],jean:10,jet_r:[4,5],john:10,joint:[4,8],juggl:4,jump:[6,9],just:[0,4,5,7,8,9],justifi:5,k_grid:7,kannappan:10,kbe:[],kei:4,kfe:[],kind:6,know:[5,6,8,9,10],known:[5,7],kolmogorov:[1,5,8,9],label:[5,6,7],lack:[3,9],lambda:[3,4,5,6,7,8,9],larg:[3,5,6,7],largest:6,last:[0,1,3,5,6,9],later:[3,4,5,6],law:5,ldot:[5,6,7],lead:[3,4,5,8,9],learn:[4,5,7],least:[0,4,9],leav:[4,8],lectur:[1,3,4,5,6,7,8,9],left:[3,4,6,9],legend:[5,6,7],lemma:[1,4,7,8,9],len:7,length:[1,5,7],leq:[0,1,3,5,6,7,9],less:[4,8],let:[0,1,3,4,5,6,7,8,9],level:[1,4,5],lg16:[5,10],lig10:[5,10],liggett:10,light:9,like:[1,4,5,6],likewis:9,lim_:[1,4,5,6,9],liminf_:0,limit:[1,4,5,6],linalg:[3,4,5,8],line:[6,7],linear:[3,9],link:[0,7,9],linop:1,linopel:9,linspac:[4,5,6],list:[2,3],littl:5,loc:7,locat:1,look:[3,5,9],lose:3,low:3,lower:7,m_t:7,made:[1,3],main:[3,4,9],major:1,majortick:4,make:[0,1,5,6],mani:[0,1,3,5,6,7],map:[1,3,4,5,8,9],mapsto:[1,3,4,5,6,9],marker:7,markov:[0,1,3,4,6,7,8,10],markov_prop:0,markovian:5,martingal:10,mass:3,match:6,materi:1,mathbb:[3,4,5,7,9],mathbf:5,mathemat:[3,5,6,10],matplotlib:[0,3,4,5,6,7,8],matric:[1,4,5,8],matrici:9,matrix:[0,1,3,5,9],max:7,max_it:[3,7],max_val:7,max_x:[3,4],mean:[3,5,6,7],measur:5,mechan:8,media:10,memoryless:[1,5,7],mental:5,method:8,might:[3,5,7,9],mild:[4,5,9],milton:10,min:[3,5],minor:[3,8],minu:4,model:4,modest:7,modif:4,modifi:3,more:[1,3,4,5,6,9],moreov:[1,4,5,6,7,8,9],most:[0,1,4,5],motion:10,move:[4,5,8],mpl_toolkit:[0,4,5],mplot3d:[0,4,5],mtk:0,multipl:[1,5,8,9],must:[0,3,4,5,6],myst_nb:5,n_r:[5,7],n_t:[5,7,9],n_val:7,name:[3,7],natur:[1,4,5],necessarili:7,need:[1,3,4,5,6,9],network:10,newton:3,next:[0,1,3,5,6,7,8,9],njit:[0,3,4,5,6,7,8],none:7,nonexplos:9,nonneg:[3,4,5,6,7,8],nontrivi:[4,7],nor98:[7,9,10],norm:[0,1,9],normal:5,norri:10,notat:[1,5],note:[3,5,6,9],noth:5,notic:[3,6],notion:[5,9],now:[0,3,4,5,6,7,8,9],num_draw:[3,7],numba:[0,3,4,5,6,7,8],number:[1,5,6,7,10],numpi:[0,3,4,5,6,7,8],obei:3,object:[4,5,8],observ:[1,3,4,5,6,7],obtain:[3,4,5,6,7,9],obviou:[5,9],obvious:[3,5],occur:[5,6,7,8,9],occurr:6,odd:10,off:[4,8],offer:6,often:[1,3,4,5],ois:10,omega:1,omit:[8,9],onc:1,one:[0,1,3,4,5,6,7,8,9],ones:[1,3,4],onli:[0,4,5,6,7,8,9],oper:[0,4,9],option:5,order:[4,5,6],ordinari:[4,5],origin:4,orthogon:[],other:[1,3,4,5,6,7,8,9],otherwis:[0,3,5,8,9],our:[1,3,4,5,6,7,8,9],out:[0,1,4,5,9],outcom:[5,6],outflow:4,output:[3,5],outsid:9,over:[1,4,5,6],own:5,p_0:[1,3,4,5,9],p_h:[1,4],p_t:[0,1,3,4,5,8,9],pair:[1,4,5,8],palaniappan:10,par08:[7,10],paramet:[3,5,6,7],parameter:[4,6],pardoux:10,pareto:[5,6],parsimoni:5,part:3,particular:[1,3,4,6,7,8,9],pass:6,past:5,path:[1,3,4,5,9],pde:[1,3],peopl:6,perspect:[4,8],phenomenon:6,phi:[5,9],pi_t:5,pick:[0,1,5,9],pin:4,place:[1,5],plan:3,planck:4,plot:[5,6,7],plot_distribution_dynam:5,plt:[0,3,4,5,6,7,8],plu:5,plug:5,pmatrix:[4,9],pmf:[3,5,7],point:[0,1,5,7,8],pointwis:[1,4,9],poisson:[1,6,9],poisson_v:7,poly3dcollect:[0,4],poor:6,posit:[0,3,5,6,8],possess:5,possibl:0,postmultipli:4,power:[1,5,6,9],practic:5,prasanna:10,preced:[5,8],precis:[1,5],predict:[5,6],premultipli:[4,9],present:3,press:10,previou:[1,3,5,6,7,9],previous:[4,5,9],primit:[3,5],principl:8,prior:5,probabilist:[3,9],probabl:[0,3,4,5,6,7,8,9,10],problem:[1,4,5,9],problemat:1,procedur:[],proceed:6,process:[1,3,6,9,10],prod_:[],produc:[5,8],product:[0,1,3,5,9],project:[4,5],proof:[0,1,3,4,5,6,7,8,9],proper:6,properli:1,properti:[1,4,9],proport:[6,7],prove:[0,1,3,4,5,6,8,9],provid:[1,4,5,7,8],psi:[0,3,4,5],psi_0:[4,5,8],psi_:4,psi_t:[3,4,5],purchas:5,pure:3,push:4,pyplot:[0,3,4,5,6,7,8],qquad:[3,4,5,6,7,8,9],quad:[1,3,4,6,7],quantecon:[0,3,4,5,6,7,8],question:[3,4,5],queue:1,quickli:9,quit:[1,5],rais:6,random:[3,5,6,7],rang:[3,4,5,6,7],rate:[3,4,6,7,8,9],rather:[1,3,5,6],ration:6,rational:7,rcS:5,reach:3,reader:[1,5],real:6,realiz:7,realli:4,rearrang:3,reason:3,reassur:3,recal:[1,3,4,5,8,9],recov:[1,5],red:6,reduc:5,refer:[],regard:[3,4,5,6],regardless:6,regular:[],rel:[1,6,9],relat:[4,7,9],relationship:7,relax:[3,5],remain:4,remark:[6,7],repeat:[3,4,7,9],rephras:4,replac:5,report:5,repres:[1,4,5,9],represent:[1,3],reproduc:5,requir:[1,4,6,9],respect:[3,5,6],rest:4,restart:[5,6],restat:6,restaur:7,restrict:[1,3,4,9],result:[0,1,3,4,5,6,8,9],revers:1,review:5,rewrit:3,right:[0,1,3,4,5,6,7,9],roulett:6,row:[3,4,5,6,9],rr_:[1,3,5,6,8,9],rule:4,run:[5,9],s29:7,sahoo:10,sai:[0,1,4,5,6,9],said:[5,6],same:[1,3,4,5,6,7,9],sampl:[6,7],sample_s:7,satisfi:[1,3,4,5,6,7,8,9],saw:[1,8,9],scalar:[1,3,9],scale:[3,5,6,7],scatter:4,scenario:9,scienc:10,scientist:3,scintcon:[],scipi:[0,3,4,5,6,7,8],searchsort:5,second:[1,3,4,6,7,8,9],section:[1,4,5,6,7,9],see:[0,1,3,4,5,6,7,8,9],seed:[5,6,7],seek:0,seem:[3,5],seen:[1,3,4,6],select:[],self:6,semigroup:[0,4,8],send:4,sens:[1,4,5,6],separ:7,sequenc:[0,1,3,5,6,7],set:[0,1,3,4,6,7,8,9],set_facecolor:4,set_pad:4,set_titl:7,set_xlabel:5,set_xtick:7,set_xticklabel:[4,7],set_ylabel:5,set_ytick:7,set_yticklabel:4,set_zticklabel:4,shall:[1,6],shape:[6,7],shift:5,shine:3,shop:[6,7],shorter:0,shortest:0,should:1,show:[1,3,4,5,6,7,8,9],shown:[0,5,6,7],side:[3,5,6,7,9],sigma:5,sim:[5,6,7],sim_path:5,similar:[1,5,7],similarli:6,simpl:[1,5,8,9],simplest:[5,7],simplex:4,simpli:5,simplifi:[3,5,6],simul:[3,4,6,7],sinc:[0,1,3,5,6,7,9],situat:[],size:[5,7],sk11:[1,10],slightli:1,small:[0,3,5,6,7],soc:10,solut:[],solv:[1,3,4,5,9],some:[0,1,3,4,5,6,7,8,9],someth:1,sometim:[4,7,8],son:10,soon:[1,5,6],sound:3,space:[0,3,6,8,9],special:[1,3,5,6,7,9],specif:[1,5],specifi:[1,6],spin:6,springer:10,squar:4,stage:5,stai:8,standard:5,start:[0,1,3,4,5,6,7],stat:[3,5],state:[0,1,4,6,7,8],statement:[0,1,4,5,6,7,9],stationar:5,stationari:5,steinhau:1,step:[0,3,5,7,8],step_siz:5,still:[3,9],stochast:[5,7,10],stock:5,stop:[6,7],store:5,str13:[3,10],strictli:[0,7],strong:5,strongli:4,stroock:10,structur:5,studi:[1,3],submultipl:1,subplot:[3,5,6,7],subset:[6,7],succe:6,suffic:1,suffici:[0,6],suggest:[0,1,4],suitabl:4,sum:[0,1,3,4,5,9],sum_:[0,1,3,4,5,6,7,8,9],sum_i:[3,5,7,8,9],sum_n:9,sum_x:[5,9],sum_z:[3,5,9],summabl:9,summar:[5,8],summari:5,sup_:[1,5],sup_x:[5,9],suppli:[3,6,9],support:[5,6,7],suppos:[0,3,4,5,6,7,8,9],supremum:5,sure:[6,9],swap:5,systemat:4,t_0:6,t_1:[5,6],t_grid:[6,7],t_i:[5,6,7],t_k:[5,7],t_n:1,t_tick:7,take:[0,1,3,4,5,6,7,9],taken:[3,4,5],tan:9,tau:[1,3,5],technic:[1,3,4,5],tell:[1,4,6],term:[0,3,8],terminolog:[1,4,5],test:7,text:[0,1,2,3,4,5,6,7,8,9],than:[1,3,4,5,6],thei:[4,5,7,9],them:[3,5],theorem:[1,4,5,6,7,8,9],theoret:[1,8],theori:[1,4,5,6,9],theta:[6,7],theta_n:7,thi:[0,1,3,4,5,6,7,8,9],think:[1,3,5,6],third:6,thm:0,thoma:10,those:[1,5,6],though:1,thought:6,three:[1,3,4,6],threshold:6,through:[4,5,8,9],throughout:[1,4],thu:[5,8],tight_layout:7,time:[0,1,3,6,8,9,10],togeth:4,tonelli:5,total:5,tq1:3,track:5,trajectori:1,transit:4,translat:9,treat:5,treatment:[1,4,9],tri:[4,6],triangl:[0,9],triangleright:[],trickier:[],trivial:3,turn:[1,5,6],two:[5,6,8],typic:[1,5],u_0:1,u_h:1,u_i:0,u_t:1,ucsgec:[],unbound:[],unchang:[4,9],uncondit:6,uncount:5,under:[0,1,4,5,9],understand:[4,5],understood:[4,5],uniform:[1,5,7],uniformli:[],uniqu:[4,5],unit:[1,3,4,5],unit_simplex:4,univers:[4,10],unless:1,unlik:[3,6],unlimit:6,until:[5,7],updat:[3,4,5],upon:5,upper:1,use:[0,3,4,5,6,7,8,9],used:[1,3,4,7],useful:5,uses:[1,4],using:[1,3,4,5,6,7],usmg:[],usual:[1,4,5],v_0:3,v_i:7,v_t:3,val:7,valid:[5,8,9],valu:[1,3,5,7,9],valuabl:8,vari:3,variabl:[3,5,6,7],vdot:9,vector:[0,3,5,9],veri:[1,5,6,8],verif:9,verifi:[1,3,5,7,8,9],version:[0,1,3,5],vertex:4,vertic:7,via:[1,3,4,5,7,8,9],view:[1,7],view_init:4,visit:7,visitor:7,visual:[4,7],vline:7,volum:10,vtx:4,w_1:[3,7],w_i:[6,7],w_k:[3,5,7],w_n:7,wai:[1,3,5,6,7,8,9],wait:[3,4,5,6,7],wal12:[5,10],walsh:10,want:4,weaker:0,websit:7,well:[1,3,4,5,7,9],were:3,what:[1,4,5,6,7,9],wheel:6,when:[0,1,3,4,5,6,7,8,9],whenev:[3,4,5,6,9],where:[1,3,4,5,7,8,9],which:[1,3,4,5,6,7,9],whole:5,why:1,width:[3,5],wilei:10,wish:[1,5],without:[3,6],word:[3,5,6,7,9],work:[1,3,4,8,9],worth:[3,4,9],would:[1,5],write:[0,1,3,4,5,6,9],written:[3,4],wrong:6,x_0:[1,3,4,5],x_1:[4,5],x_2:5,x_j:5,x_k:5,x_n:[4,5],x_r:5,x_t:[0,1,3,4,5,8,9],x_val:4,xaxi:4,xlabel:[3,5,7],xlim:4,xtick:[4,7],xticklabel:7,y_0:[3,5],y_1:[3,5],y_i:5,y_k:[3,5],y_t:3,y_val:[4,5],yaxi:4,yet:9,yield:[0,3],ylabel:5,ylim:4,yoshida:9,you:[1,3,4,5,6,7,8,9],your:5,ytick:[4,7],z_0:[],z_1:0,z_2:0,z_i:0,z_k:[],z_m:0,z_val:4,zaxi:4,zdir:5,zero:[1,3,4,5,6,7,9],zip:7,zlim:4,ztick:4,zz_:[5,6,7,9]},titles:["Stationarity and Ergodicity","Semigroups and Generators","Continuous Time Markov Chains","Kolmogorov Backward Equation","The Kolmogorov Forward Equation","The Markov Property","Memoryless Distributions","Poisson Processes","A Probabilistic View","UC Markov Semigroups","Bibliography"],titleterms:{"case":[4,5,8,9],"function":1,ODEs:4,The:[1,3,4,5,6,7,8,9],algorithm:[3,8],applic:3,asymptot:0,back:[],backward:[3,4],beyond:9,bibliographi:10,bound:9,calculu:1,can:7,canon:5,chain:[2,3,4,5,8],character:[1,6,7],check:3,comput:3,condit:9,conserv:8,constant:5,construct:[5,8],continu:[1,2,4,5],count:7,countabl:5,curv:1,depend:3,differ:4,differenti:[1,3],discret:[4,5],distribut:[0,4,5,6],dynam:5,embed:[3,5],equat:[3,4],ergod:0,exampl:5,exercis:[1,3,4,5,6,7,8,9],exponenti:[1,3,6,7],extend:5,failur:[5,6],finit:[5,9],flow:5,forward:4,from:[3,4,6,8],gener:1,geometr:6,gillespi:8,hold:7,implic:5,impos:5,increment:7,independ:7,integr:3,intens:[3,8,9],inventori:[3,5],irreduc:0,joint:5,jump:[3,4,5,7,8],kolmogorov:[3,4],linear:[1,4],map:[],markov:[2,5,9],matric:9,matrix:[4,8],memoryless:6,model:[3,5],motiv:[1,3],necessari:9,notat:9,oper:1,overview:[0,1,3,4,5,6,7,8,9],pair:9,paus:7,poisson:[5,7],preliminari:1,preserv:4,probabilist:[5,8],process:[5,7],properti:[3,5,6,7],rate:5,refer:10,represent:5,restart:7,restrict:5,review:4,semigroup:[1,3,5,9],set:5,shift:4,simul:[5,8],solut:[1,3,4,5,6,7,8,9],space:[1,4,5],stabiltii:0,state:[3,5,9],stationar:0,stationari:[0,7],suffici:9,sum:6,summari:4,terminolog:9,time:[2,4,5,7],transit:[3,5],uniformli:1,uniqu:[0,3,7,9],valu:4,vector:4,view:8}}) \ No newline at end of file +Search.setIndex({docnames:["ergodicity","generators","intro","kolmogorov_bwd","kolmogorov_fwd","markov_prop","memoryless","poisson","uc_mc_semigroups","zreferences"],envversion:{"sphinx.domains.c":1,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":1,"sphinx.domains.index":1,"sphinx.domains.javascript":1,"sphinx.domains.math":2,"sphinx.domains.python":1,"sphinx.domains.rst":1,"sphinx.domains.std":1,"sphinx.ext.intersphinx":1,sphinx:56},filenames:["ergodicity.md","generators.md","intro.md","kolmogorov_bwd.md","kolmogorov_fwd.md","markov_prop.md","memoryless.md","poisson.md","uc_mc_semigroups.md","zreferences.md"],objects:{},objnames:{},objtypes:{},terms:{"100_000":3,"10_000":[6,7],"1_000":6,"1e5":7,"\u0144":9,"\u03b1":[3,5],"\u03b3":3,"\u03b8":7,"\u03b8_val":7,"\u03bb":[3,5,6,7],"\u03c8":[0,3,4,5],"\u03c8_0":[3,5],"\u03c8_00":[0,4],"\u03c8_01":[0,4],"\u03c8_02":[0,4],"\u03c8_star":[],"\u03c8_t":3,"abstract":[1,4,5],"break":5,"case":[0,1,3,6,7],"class":[0,6],"final":[3,5],"function":[0,3,4,5,6,7,8,9],"import":[0,1,3,4,5,6,7,8],"long":[0,5,6],"new":[3,5,7],"return":[0,1,3,4,5,6,7],"short":2,"switch":5,"true":[0,1,3,4,5,7],"try":[3,4,5,7,8],"while":[1,3,4,5,6,7,8],And:4,Are:6,But:[0,4,6,8],For:[0,1,3,4,5,6,7,8],Its:5,M_s:7,N_s:[5,7],Not:5,ODE:[0,1,4],ODEs:[1,3],One:[1,3,5,7],P_s:[0,3,5],Such:[0,1],That:[5,7],The:[0,2],Then:[0,5,6,7,8],There:[0,1,2,4,5],These:[4,5,8],U_s:1,Using:[1,4,6,7],V_s:3,Will:0,With:[0,5],X_s:5,__call__:6,__init__:6,_axinfo:[0,4],a_n:6,abbrevi:[1,5],abl:1,about:[0,1,2,3,5,6],abov:[1,3,4,5,6,7,8],absorb:[0,8],acceler:2,access:0,accord:[4,5],across:6,actual:3,adam:9,adapt:4,add:[0,5],add_collection3d:[0,4],add_subplot:[0,4,5],added:[],addit:[1,5,7,8],address:0,admit:8,adopt:[1,3],affirm:5,after:[0,1,3,4,5,6],again:[0,1,4,5,6,7,8],agre:[5,7,8],aim:[0,1,2,5,8],air:0,alcohol:6,algebra:[1,5],algorithm:[0,4,6,8,9],align:[0,3,5],all:[0,1,2,3,4,5,6,7,8],allow:[1,3,5],along:[0,3,4,5,8],alpha:[0,1,3,4,5,6,7],alreadi:[4,5,6,7],also:[1,3,4,5,6,7,8],altern:[5,7,8],although:[4,5,6,8],alwai:[1,3,4,5,8],american:9,amount:[2,3,5,6],analog:[0,1,3,4,5,8],analysi:[6,9],analyt:[3,4,8],analyz:5,andrzej:9,angl:[0,4],ani:[0,1,3,4,5,6,7,8],anoth:[1,3,5,6,7,8],answer:[3,5,7],aperiod:0,app19:[1,9],appeal:[7,8],append:[0,4,5],applebaum:9,appli:[0,3,4,5,7,8],appliat:8,applic:[0,1,2,4,5,8,9],approach:[3,4,5,9],appropri:1,approx:[6,7],approxim:[3,7],arang:[3,5,7],arbitrari:[0,1,3,5,7,8],arbitrarili:0,argu:4,argument:[1,3,4,5,8],aris:[1,8],arrai:[0,4,5],arriv:[0,3,5,6,7],art3d:[0,4],ask:[0,3,5,7,8],aspect:9,assert:4,associ:[0,1,3,4,7,8],assum:[0,1,2,3,4,5,8],assumpt:[3,4],attempt:2,attent:8,attest:6,attribut:0,author:2,automat:5,autonom:4,averag:0,avoid:[1,4],awai:8,axes3d:[0,4,5],axes:7,axi:[0,4],b_n:6,back:[0,4,5],backward:[1,8],banach:[1,2,8],bar:[0,3,5],basic:[1,2],becaus:[0,1,3,4,5,6,7,8],becom:[3,5,6,7],been:[4,5],befor:[0,4,5],begin:[0,3,4,5,6,8],behavior:0,behind:7,being:[1,4],below:[0,1,3,4,5,6,7,8],bernoulli:5,bet:6,beta:1,better:1,between:[0,1,2,3,4,5,6,7,8],beverag:6,big:1,binari:[5,7],binom:[3,5,6,7],binom_v:7,binomi:[3,5,7],biolog:9,birth:0,black:[0,6],bob05:[1,9],bobrowski:9,book:[2,5],both:[0,4,5,6,8],bound:[0,1,5],bounded:[1,8],bridg:2,brief:8,brownian:9,build:[2,5,8],built:8,busi:9,c_0:[1,8],calcul:[0,3,5],calculu:9,call:[0,1,3,4,5,6,7,8],cambridg:9,can:[0,1,2,3,4,5,6,8],cannot:[1,5],captur:0,care:[1,4],carri:[4,5],cartiesian:5,casino:6,cauchi:1,cdf:[6,7],cdot:[0,1,3,4,5,7,8],center:7,certain:4,certainli:1,chain:[0,1,6,7,9],challeng:1,chang:[1,3,4,5,6,8],chao:9,chapman:5,chapter:1,character:0,charcter:8,check:[1,4,5,8],choic:[0,8],choos:8,chosen:[4,5],circumv:1,claim:[1,3,6,7,8],clarifi:[1,2,3,4,5,6],clear:[3,7],clearer:5,clearli:8,close:[5,6],code:[3,4],coeffici:3,coincid:1,collect:[5,8],colon:[0,3,8],color:[0,4,5],column:4,combin:[0,1,2,3,4,8],come:[1,6],comfort:2,common:[0,5,6],commonli:4,commun:0,compar:[6,7],comparison:4,compil:2,complet:[1,5,6,8],complex:3,complic:1,composit:[1,8],comput:[5,8,9],concept:0,concern:[0,1],conclud:[4,5,7,8],concret:5,condit:[0,3,4,5],confirm:[0,3,4,6,8],confus:5,connect:[0,1,4,5,6,7,8],consequ:[0,7],consid:[0,1,4,5,6,8],consist:[3,5,6,8],constant:[0,1,3,4,8],construct:[0,3,4,6,7],consum:7,contabl:8,contain:[0,8],context:4,continu:[0,3,6,7,8,9],contract:[],contradict:[0,8],contrari:0,conveni:[3,5],convent:5,converg:[0,1,6,7,8],convergence_plot:4,convers:[4,6,8],convert:3,convinc:3,cool:[0,4,5],corollari:[0,5],correct:[4,5,7],correspond:[0,4,5,8],could:5,countabl:[0,4,8],cours:5,cousin:2,cover:[2,5,8],crc:9,creat:[4,5],critic:0,cross:[5,6],cumsum:7,curios:2,current:[4,5,6,8],custom:[0,3,5,6,7],d_t:0,daniel:9,darkblu:[0,4],data:5,date:[4,5],david:9,ddot:0,deal:2,decompos:8,decreas:6,deduc:8,deep:7,def:[0,3,4,5,6,7],defer:4,defin:[1,3,4,5,6,7,8],definit:[1,3,4,5,6,7,8],delai:5,delet:5,demonstratio:9,denot:[1,3,4,5,6],densiti:6,depend:[0,4,5,6,7,8],depthshad:[0,4],deriv:[1,3,4,8],describ:[3,4,5],descript:[0,1,3,4,5],design:0,desir:5,despit:1,detail:[1,5,7,8],determin:[5,8],determinist:3,develop:5,diagon:[4,8],differ:[3,5,6,7],differenti:[4,8],difficult:[3,5,7],diffus:[1,4],dimension:[1,3,4,5,8],direct:[0,1,5],directli:[0,1,4,8],discourag:6,discret:[0,1,6,7,8],discuss:[0,1,2,3,4,5,6,7,8],disjoint:0,displai:[0,5],distinct:0,distract:4,distribut:[1,3,7,8],doe:[0,1,5],domain:1,domin:8,done:[1,3],dot:0,dougla:9,down:[3,4,5],downarrow:[1,4],draw:[3,5,6,7],draw_nt:7,draw_nt_sampl:7,draw_x:[3,6],drawn:[2,3],driven:1,dtype:3,due:[0,3,5,6,7],dynam:[4,9],e_1:0,e_2:0,each:[0,1,3,4,5,6,7,8],earli:[4,5],earlier:[3,7,8],easi:[0,1,3,4,5,8],easier:[0,3,8],easili:[1,3,4,8],econom:[2,5,9],economist:5,effici:5,either:[5,6,7],elaps:6,eleg:5,element:[0,1,3,4,5,8,9],ell:5,ell_1:[0,1,8],els:[3,5,6],elsewher:5,embed:[0,3,8],emphasi:4,empir:[6,7],empirical_exceed:6,emploi:4,empti:[3,6,7],empty_lik:3,encourag:3,end:[0,3,4,5,6,7,8],enjoi:1,enough:[0,5,8],ensur:8,enter:6,entir:[1,3,5,6],entri:[3,6],environ:7,epsilon:0,equal:[1,3,4,5,8],equat:[1,5,8,9],equilibrium:0,equival:[0,1,4,5,7,8],erlang:[6,7],essenti:8,establish:[0,1,8],etc:[7,8],etienn:9,evalu:[3,5,7],even:[1,5],evenli:6,event:6,everi:[0,1,4,5,8],everywher:[0,1,8],evolut:[1,4,5],evolv:[0,4,5],exact:[4,8],exactli:5,examin:[4,7],exampl:[0,1,3,4,6,7,8],exceed:6,excel:1,except:0,exist:[0,1,4,5,6,7,8],exp:[0,3,5,6,7],expect:[3,4,5,7,8],experi:[3,7],explicitli:[3,4,5],explod:[1,8],explor:[1,5],explos:8,expm:[0,3,4,5],exponenti:[4,5,8],exposit:5,express:[1,5,7,8],extend:4,extens:[1,5],fact:[0,1,3,4,5,6,7,8],factori:[6,7],fail:[1,3,5,6],fall:[5,6],fals:[0,4,5],famili:[5,8],familiar:[1,2,5],far:[5,6],fascin:2,fast:2,featur:7,feel:6,ff_:5,field:2,fifth:6,fig:[0,3,4,5,6,7],figsiz:[0,4,5,7],figur:[0,3,4,5,6,7],file:0,financ:9,find:[],finer:6,finit:[0,1,3,4,7],firm:5,first:[0,1,3,4,5,6,7,8],fit:[1,6,7],five:6,fix:[0,1,3,4,5,6,7,8],flatten:7,flow:[0,1,4,8],flow_fig:5,flow_plot:[0,4],focu:[1,2,3,8],focus:5,foguel:[],fokker:4,follow:[0,1,3,4,5,6,7,8],fontsiz:[0,4,7],foral:[1,6],forev:8,forget:6,form:[1,3,5,6,8],formal:5,formula:5,fortun:[5,8],forward:[1,5,8],found:[0,1,2,4,7],four:6,frac:[0,1,3,4,5,6,7,8],fractal:9,fraction:6,framework:1,fran:9,free:6,freeli:7,from:[1,2,5,7],full:[1,3,5],fulli:1,fundament:[4,5],further:[3,5,6,7],futur:[5,6],gall:9,gamma:3,gap:2,gener:[3,4,5,7],genom:9,geometr:[3,5,7],geq:[0,1,3,4,5,6,7,8],get:[0,4,5,6],give:[0,1,3,4,5,8],given:[0,1,3,4,5,6,7,8],global:0,glue:5,goe:[5,8],good:[6,7],govern:5,great:3,grid:[0,4,5,6,7],grow:[0,8],guess:3,h_n:1,halv:6,hand:[1,3,5,6,7,8],hard:[1,3,8],harder:3,has:[0,1,3,4,5,6,7,8],hat:[3,4,5,7,8],have:[0,1,2,3,4,5,6,7,8],haven:4,help:[0,2,3,5,6,7],henc:[0,1,3,4,5,6,7,8],henceforth:[4,5],here:[0,2,3,4,5,6,7,8],high:1,higher:0,hill:8,hint:[1,4,5],histogram:[3,5],histori:[5,6],hline:7,hold:[0,1,3,4,5,6,8],homogen:[4,5],hope:8,hospit:0,hot:[4,5],hour:6,how17:[7,9],how:[1,4,5,6,8],howard:9,howev:[1,8],hypothesi:5,idea:[0,1,2,3,4,5,6,7,8],ideal:1,ident:[1,3,5,8],identif:8,identifi:[0,1,4,5,8],iff:3,iid:[3,5,6,7],illustr:[5,6,7],imagin:6,immedi:[0,3,4,5,8],impli:[0,1,3,5,8],implic:4,implicitli:3,impos:4,improv:7,includ:5,increas:[6,7],increment:5,inde:[0,1,3,5,7],indepen:7,independ:[3,5,6],independent_draw:3,index:5,indic:[1,4,5,7],indistinguish:6,individu:0,induc:8,induct:[5,7],inequ:[0,8],infin:[0,8],infinit:[0,1,2,4,5,8],infinitesim:[0,1,3,4],inflow:4,inform:[3,5,7],infti:[0,1,3,4,5,6,7,8],ingredi:0,initi:[0,1,3,4,5,8],input:[3,5],insert:[5,7,8],insight:3,insist:5,instead:[3,7],instinct:6,int64:3,int_0:3,integ:[5,6,7],integr:2,intenintens:8,intens:[0,1,4,5],interest:[1,5,6],interpret:[1,5],interv:[5,6,7],introduc:[1,5,8],introduct:[1,2,5,6,9],intuit:[0,2,3,4,5,7,8],invari:4,inventori:4,invert:1,investig:[3,7],involv:5,isaac:3,issu:[1,8],iter:5,its:[0,1,5,6,7],itself:[0,1,4,5,8],j_0:[3,5,7],j_1:[0,3],j_2:0,j_k:[0,3,5,7],j_n:[6,7],j_val:5,jame:9,jean:9,jet_r:[0,4,5],john:[2,9],joint:4,juggl:[0,4],jump:[0,6],jupyt:2,just:[2,4,5,7,8],justifi:5,k_grid:7,kami:9,kannappan:9,katarzyna:9,kei:[0,4],kind:[0,6],know:[0,5,6,8,9],knowledg:2,known:[5,7],kolmogorov:[1,5,8],label:[5,6,7],lack:[3,8],lambda:[0,3,4,5,6,7,8],lambda_0:0,lambda_1:0,lambda_2:0,lambda_j:0,lambda_k:0,larg:[3,5,6,7],largest:6,lasota:9,last:[0,1,3,5,6,8],later:[2,3,4,5,6],law:5,ldot:[5,6,7],lead:[0,3,4,5,8],learn:[2,4,5,7],least:[0,2,4,8],leav:[0,4,8],lectur:[0,1,2,3,4,5,6,7,8],left:[0,3,4,6,8],legend:[5,6,7],lemma:[0,1,4,7,8],len:7,length:[0,1,5,7],leq:[0,1,3,5,6,7,8],less:4,let:[0,1,3,4,5,6,7,8],level:[1,4,5],lg16:[5,9],lig10:[5,9],liggett:9,light:8,like:[1,4,5,6],likewis:8,lim_:[1,4,5,6,8],liminf_:0,limit:[1,4,5,6],linalg:[0,3,4,5],line:[0,6,7],linear:[0,2,3,8,9],link:[0,7,8],linop:1,linopel:8,linspac:[0,4,5,6],list:3,littl:5,lm94:[0,9],loc:7,locat:1,look:[0,2,3,5,8],lose:3,low:3,lower:7,m_t:7,mackei:9,made:[1,3],main:[3,4,8],mainli:2,major:1,majortick:[0,4],make:[1,5,6],mani:[0,1,2,3,5,6,7],manufactur:0,map:[1,3,4,5,8],mapsto:[1,3,4,5,6,8],marker:7,markov:[0,1,3,4,6,7,9],markovchain:0,markovian:5,marta:9,martingal:9,mass:3,match:6,materi:1,mathbb:[3,4,5,7,8],mathbf:5,mathemat:[2,3,5,6,9],mathematica:9,matplotlib:[0,3,4,5,6,7],matric:[1,4,5],matrici:8,matrix:[0,1,3,5],matter:0,max:7,max_it:[3,7],max_val:7,max_x:[3,4],mean:[0,3,5,6,7],measur:5,mechan:8,media:9,memoryless:[1,5,7],mental:5,method:0,michael:9,might:[3,5,7,8],mild:[4,5,8],milton:9,min:[3,5],minor:[3,8],minu:4,mit:9,model:[0,4,9],modest:7,modif:4,modifi:3,moment:0,month:0,more:[1,2,3,4,5,6,8],moreov:[0,1,4,5,6,7,8],most:[0,1,4,5],motion:9,move:[0,4,5,8],mpl_toolkit:[0,4,5],mplot3d:[0,4,5],mtk:[],multipl:[1,5,8],must:[0,3,4,5,6],myst_nb:5,n_r:[5,7],n_t:[5,7,8],n_val:7,name:[3,7],natur:[1,4,5],necessarili:7,need:[0,1,3,4,5,6,8],network:9,newton:3,next:[0,1,3,5,6,7,8],njit:[0,3,4,5,6,7],nois:9,none:7,nonexplos:8,nonneg:[3,4,5,6,7,8],nontrivi:[4,7],nor98:[7,8,9],norm:[0,1,8],normal:5,norri:9,notat:[1,5],note:[0,3,5,6,8],noth:5,notic:[3,6],notion:[5,8],now:[0,3,4,5,6,7,8],num_draw:[3,7],numba:[0,2,3,4,5,6,7],number:[1,5,6,7,9],numpi:[0,2,3,4,5,6,7],obei:3,object:[4,5,8],observ:[0,1,3,4,5,6,7],obtain:[0,3,4,5,6,7,8],obviou:[0,5,8],obvious:[3,5],occur:[5,6,7,8],occurr:6,odd:9,off:[4,8],offer:6,often:[0,1,3,4,5],ois:9,omega:1,omit:8,onc:1,one:[0,1,2,3,4,5,6,7,8],ones:[1,3,4],onli:[0,4,5,6,7,8],oper:[0,2,4,8,9],oppos:0,optim:0,option:5,order:[4,5,6],ordinari:[4,5],origin:[0,4],other:[1,3,4,5,6,7,8],otherwis:[0,3,5,8],our:[0,1,2,3,4,5,6,7,8],out:[0,1,4,5,8],outcom:[5,6],outflow:4,output:[3,5],over:[0,1,4,5,6],own:5,p_0:[1,3,4,5,8],p_1:0,p_h:[0,1,4],p_t:[0,1,3,4,5,8],pace:2,pair:[0,1,4,5],palaniappan:9,par08:[7,9],paramet:[0,3,5,6,7],parameter:[4,6],pardoux:9,pareto:[5,6],parsimoni:5,part:3,particular:[1,3,4,6,7,8],pass:6,past:5,path:[1,3,4,5,8],patient:0,pde:[1,3],peopl:6,period:0,perspect:[4,8],phenomenon:6,phi:[0,5,8],pi_t:5,pich:9,pichorrtkaminska12:[0,9],pick:[0,1,5,8],pin:4,place:[1,5],plan:3,planck:4,plenti:2,plot:[5,6,7],plot_distribution_dynam:5,plt:[0,3,4,5,6,7],plu:5,plug:5,pmatrix:[0,4,8],pmf:[3,5,7],point:[0,1,5,7,8],pointwis:[1,4,8],poisson:[1,6,8],poisson_v:7,poly3dcollect:[0,4],poor:6,posit:[0,3,5,6,8],possess:5,postmultipli:4,power:[0,1,5,6,8],practic:5,prasanna:9,preced:[5,8],precis:[1,5],predict:[5,6],preliminari:2,premultipli:[4,8],present:[2,3],press:9,previou:[1,3,5,6,7,8],previous:[4,5,8],primit:[3,5],principl:8,prior:5,probabilist:[3,8],probabl:[0,3,4,5,6,7,8,9],problem:[1,4,5,8],problemat:1,proceed:6,process:[0,1,3,6,8,9],prod_:0,produc:[5,8],product:[0,1,3,5,8],project:[0,4,5],proof:[0,1,3,4,5,6,7,8],proper:6,properli:1,properti:[1,4,8],proport:[6,7],proposit:0,prove:[0,1,3,4,5,6,8],provid:[1,2,4,5,7,8],psi:[0,3,4,5],psi_0:[4,5],psi_:4,psi_t:[3,4,5],purchas:5,pure:[0,3],push:4,pyplot:[0,3,4,5,6,7],python:2,qquad:[3,4,5,6,7,8],quad:[1,3,4,6,7],quantecon:[0,2,3,4,5,6,7],question:[3,4,5],queue:[0,1],quickli:8,quit:[1,5],rais:6,random:[3,5,6,7],rang:[0,3,4,5,6,7],rate:[0,3,4,6,7,8],rather:[0,1,2,3,5,6],ration:6,rational:7,rcS:5,reach:3,reader:[1,2,5],real:6,realiz:7,realli:4,rearrang:3,reason:3,reassur:3,recal:[0,1,3,4,5,8],recommend:2,record:0,recov:[1,5],red:6,reduc:5,regard:[3,4,5,6],regardless:6,rel:[1,2,6,8],relat:[4,7,8],relationship:7,relax:[3,5],remain:4,remark:[6,7],repeat:[3,4,7,8],repeatedli:0,rephras:4,replac:5,report:5,repres:[1,4,5,8],represent:[1,3],reproduc:5,request:0,requir:[1,2,4,6,8],research:2,respect:[3,5,6],rest:4,restart:[5,6],restat:6,restaur:7,restrict:[1,3,4,8],result:[0,1,3,4,5,6,8],revers:1,review:[2,5],rewrit:3,right:[0,1,3,4,5,6,7,8],rigor:2,roulett:6,row:[3,4,5,6,8],rr_:[0,1,3,5,6,8],rudnicki:9,rule:[0,4],run:[0,5,8],ryszard:9,s29:7,sahoo:9,sai:[0,1,4,5,6,8],said:[5,6],same:[0,1,3,4,5,6,7,8],sampl:[6,7],sample_s:7,sargent:2,satisfi:[1,3,4,5,6,7,8],saw:[1,8],scalar:[1,3,8],scale:[3,5,6,7],scatter:[0,4],scenario:8,scheme:0,scienc:9,scientif:2,scientist:3,scipi:[0,3,4,5,6,7],searchsort:5,second:[1,3,4,6,7,8],section:[0,1,4,5,6,7,8],see:[0,1,3,4,5,6,7,8],seed:[5,6,7],seek:0,seem:[0,3,5],seen:[1,3,4,6],self:6,semigroup:[0,4,9],send:4,sens:[1,4,5,6],separ:7,sequenc:[0,1,3,5,6,7],seri:2,serv:0,server:0,servic:0,set:[0,1,3,4,6,7,8],set_facecolor:[0,4],set_pad:[0,4],set_titl:7,set_xlabel:5,set_xtick:7,set_xticklabel:[0,4,7],set_ylabel:5,set_ytick:7,set_yticklabel:[0,4],set_zticklabel:[0,4],shall:[1,6],shape:[6,7],share:2,shift:5,shine:3,shop:[6,7],should:1,show:[0,1,3,4,5,6,7,8],shown:[0,5,6,7],side:[3,5,6,7,8],sigma:5,sim:[0,5,6,7],sim_path:5,similar:[1,5,7],similarli:6,simpl:[1,5,8],simplest:[5,7],simplex:4,simpli:5,simplifi:[3,5,6],simul:[3,4,6,7],sinc:[0,1,3,5,6,7,8],size:[5,7],sk11:[1,9],ska:9,slightli:1,small:[2,3,5,6,7],soc:9,solut:2,solv:[0,1,3,4,5,8],some:[0,1,2,3,4,5,6,7,8],someth:1,sometim:[4,7],son:9,soon:[1,5,6],sound:3,space:[0,2,3,6,8],special:[1,3,5,6,7,8],specif:[1,5],specifi:[1,6],spin:6,springer:9,squar:4,sta09:[0,9],stabil:[],stabl:0,stachurski:[2,9],stage:5,stai:8,standard:5,start:[0,1,2,3,4,5,6,7],stat:[3,5],state:[0,1,2,4,6,7],statement:[0,1,4,5,6,7,8],stationar:5,stationari:5,stationary_distribut:0,steinhau:1,step:[0,3,5,7,8],step_siz:5,still:[3,8],stochast:[5,7,9],stock:5,stop:[6,7],store:5,str13:[3,9],strict:[],strictli:[0,7],strong:[0,5],strongli:4,stroock:9,structur:5,studi:[1,3],submultipl:1,subplot:[3,5,6,7],subset:[0,6,7],succe:6,suffic:1,suffici:[0,6],suggest:[0,1,4],suit:2,suitabl:[2,4],sum:[0,1,3,4,5,8],sum_:[0,1,3,4,5,6,7,8],sum_i:[0,3,5,7,8],sum_n:8,sum_x:[0,5,8],sum_z:[3,5,8],summabl:8,summar:[5,8],summari:5,sup_:[1,5],sup_x:[5,8],suppli:[3,6,8],support:[5,6,7],suppos:[0,3,4,5,6,7,8],supremum:5,sure:[6,8],swap:5,systemat:4,t_0:6,t_1:[5,6],t_grid:[6,7],t_i:[5,6,7],t_k:[5,7],t_n:1,t_tick:7,take:[0,1,3,4,5,6,7,8],taken:[3,4,5],tan:8,tau:[1,3,5],technic:[1,3,4,5],tell:[0,1,4,6],tend:0,term:[0,3],terminolog:[1,4,5],test:7,text:[0,1,3,4,5,6,7,8],than:[0,1,2,3,4,5,6],thei:[4,5,7,8],them:[2,3,5],theorem:[0,1,4,5,6,7,8],theoret:1,theori:[0,1,2,4,5,6,8,9],therefor:0,theta:[6,7],theta_n:7,thi:[0,1,2,3,4,5,6,7,8],think:[1,3,5,6],third:6,thm:[],thoma:[2,9],those:[1,2,5,6],though:1,thought:6,three:[0,1,3,4,6],threshold:6,through:[2,4,5,8],throughout:[1,4],thu:[5,8],tight_layout:7,time:[0,1,3,6,8,9],togeth:4,tonelli:5,top:0,topic:[0,2],total:5,toward:2,tq1:3,track:5,traffic:0,trajectori:[0,1],transit:4,translat:8,treat:5,treatment:[0,1,4,8],tri:[0,4,6],triangl:[0,8],trivial:3,turn:[0,1,5,6],two:[0,5,6],typic:[1,5],tyran:9,u_0:1,u_h:1,u_t:1,unchang:[4,8],uncondit:6,uncount:5,under:[0,1,4,5,8],understand:[4,5,8],understood:[4,5],uniform:[1,5,7],uniqu:[4,5],unit:[1,3,4,5],unit_simplex:[0,4],univers:[4,9],unless:1,unlik:[3,6],unlimit:6,until:[5,7],updat:[3,4,5],upon:5,upper:1,use:[0,3,4,5,6,7,8],used:[0,1,3,4,7],useful:[0,5],uses:[1,4],using:[1,2,3,4,5,6,7],usual:[1,4,5],v_0:3,v_i:7,v_t:3,val:7,valid:[5,8],valu:[1,3,5,7,8],vari:3,variabl:[3,5,6,7],vdot:[0,8],vector:[0,2,3,5,8],veri:[1,5,6,8],verif:8,verifi:[1,3,5,7,8],version:[0,1,3,5],vertex:4,vertic:7,via:[1,2,3,4,5,7,8],view:[1,7,8],view_init:[0,4],visit:7,visitor:7,visual:[4,7],vline:7,volum:9,vtx:[0,4],w_1:[3,7],w_i:[6,7],w_k:[3,5,7],w_n:7,wai:[1,3,5,6,7,8],wait:[0,3,4,5,6,7],wal12:[5,9],walsh:9,want:4,weak:0,weaker:0,websit:7,week:0,well:[1,2,3,4,5,7,8],were:3,what:[0,1,4,5,6,7,8],wheel:6,when:[0,1,3,4,5,6,7,8],whenev:[3,4,5,6,8],where:[1,3,4,5,7,8],which:[0,1,2,3,4,5,6,7,8],who:2,whole:5,why:[0,1],width:[3,5],wilei:9,wish:[0,1,2,5],without:[0,3,6],word:[3,5,6,7,8],work:[1,3,4,8],worth:[3,4,8],would:[1,5],write:[0,1,3,4,5,6,8],written:[2,3,4],wrong:6,x_0:[1,3,4,5],x_1:[4,5],x_2:5,x_j:5,x_k:5,x_n:[4,5],x_r:5,x_t:[0,1,3,4,5,8],x_val:[0,4],xaxi:[0,4],xlabel:[3,5,7],xlim:[0,4],xtick:[0,4,7],xticklabel:7,y_0:[0,3,5],y_1:[0,3,5],y_i:5,y_k:[0,3,5],y_m:[],y_t:3,y_val:[0,4,5],yaxi:[0,4],yet:8,yield:[0,3],ylabel:5,ylim:[0,4],yoshida:8,you:[0,1,3,4,5,6,7,8],your:5,ytick:[0,4,7],z_0:0,z_1:0,z_2:0,z_i:0,z_m:0,z_val:[0,4],zaxi:[0,4],zdir:5,zero:[0,1,3,4,5,6,7,8],zip:7,zlim:[0,4],ztick:[0,4],zz_:[0,5,6,7,8]},titles:["Stationarity and Ergodicity","Semigroups and Generators","Continuous Time Markov Chains","Kolmogorov Backward Equation","The Kolmogorov Forward Equation","The Markov Property","Memoryless Distributions","Poisson Processes","UC Markov Semigroups","Bibliography"],titleterms:{"case":[4,5,8],"function":1,ODEs:4,The:[1,3,4,5,6,7,8],algorithm:3,analysi:2,applic:3,asymptot:0,background:2,backward:[3,4],beyond:8,bibliographi:9,bound:8,calculu:1,can:7,canon:5,chain:[2,3,4,5,8],character:[1,6,7],check:3,code:2,comput:[2,3],condit:8,conserv:8,constant:5,construct:5,continu:[1,2,4,5],contract:0,count:7,countabl:5,curv:1,decomposit:8,definit:0,depend:3,differ:4,differenti:[1,3],discret:[2,4,5],distribut:[0,4,5,6],drift:0,dynam:5,embed:5,equat:[3,4],ergod:0,exampl:5,exercis:[0,1,2,3,4,5,6,7,8],exponenti:[1,3,6,7],extend:5,failur:[5,6],finit:[5,8],flow:5,forward:4,from:[0,3,4,6,8],further:[],gener:[0,1,8],geometr:6,hold:7,implic:5,impos:5,increment:7,independ:7,integr:3,intens:[3,8],inventori:[3,5],irreduc:0,joint:5,jump:[3,4,5,7,8],kolmogorov:[3,4],linear:[1,4],markov:[2,5,8],matric:8,matrix:[4,8],memoryless:6,model:[3,5],motiv:[1,3],necessari:8,notat:8,oper:1,overview:[0,1,3,4,5,6,7,8],pair:8,paus:7,poisson:[5,7],preliminari:1,prerequisit:2,preserv:4,probabilist:5,probabl:2,process:[5,7],properti:[3,5,6,7],rate:5,refer:9,represent:5,restart:7,restrict:5,review:4,semigroup:[1,3,5,8],set:5,shift:4,simul:[5,8],skeleton:0,solut:[0,1,3,4,5,6,7,8],solv:2,space:[1,4,5],stabil:0,stabiltii:0,state:[3,5,8],stationar:0,stationari:[0,7],strict:0,suffici:8,sum:6,summari:4,terminolog:8,time:[2,4,5,7],topic:[],transit:[3,5],uniformli:1,uniqu:[0,3,7],valu:4,vector:4,via:0}}) \ No newline at end of file diff --git a/uc_mc_semigroups.html b/uc_mc_semigroups.html index b858253..0933b97 100644 --- a/uc_mc_semigroups.html +++ b/uc_mc_semigroups.html @@ -38,7 +38,7 @@ - + @@ -113,11 +113,6 @@

      Continuous Time Markov Chains

      UC Markov Semigroups
      10. -
    10. - - A Probabilistic View - -
    11. Stationarity and Ergodicity @@ -229,8 +224,8 @@

      Continuous Time Markov Chains
    12. - - A Unique Pairing + + UC Markov Semigroups and their Generators
      • @@ -245,6 +240,33 @@

        Continuous Time Markov Chains

      13. +
    13. + + From Intensity Matrix to Jump Chain + + +
    14. Beyond Bounded Intensity Matrices @@ -275,6 +297,11 @@

      Continuous Time Markov Chains

      Exercise 4
      15. +
    15. + + Exercise 5 + +
    16. @@ -302,6 +329,11 @@

      Continuous Time Markov Chains

      Solution to Exercise 4
      17. +
    17. + + Solution to Exercise 5 + +
      18. @@ -318,15 +350,20 @@

      Continuous Time Markov Chains

      UC Markov Semigroups

      Overview

      -

      In our previous lecture we covered some of the general theory of continuous -operator semigroups.

      +

      In our previous lecture we covered some of the general theory of operator +semigroups.

      Next we translate these results into the setting of Markov semigroups.

      -

      The main aim is to give an exact one-to-one correspondence between UC -Markov semigroups and “conservative” intensity matrices on a countable set \(S\).

      +

      The Markov semigroups are defined on a countable set \(S\).

      +

      The main aim is to give an exact one-to-one correspondence between

      +
        +

      • UC Markov semigroups

        • +

      • “conservative” intensity matrices and

        • +

      • jump chains with state dependent jump intensities

        • +

      Conservativeness is defined below and relates to “nonexplosiveness” of the -associated Markov matrix.

      -

      We will also give a brief discussion of intensity matricies that fall -outside this class, along with the processes they generate.

      +associated Markov chain.

      +

      We will also give a brief discussion of intensity matricies that do not have +this property, along with the processes they generate.

      Notation and Terminology

      @@ -367,8 +404,8 @@

      Notation and Terminology(49) is a bounded linear operator on \(\ell_1\).

      Below we show how this property can be checked in applications.

      -
      -

      A Unique Pairing

      +
      +

      UC Markov Semigroups and their Generators

      Let \(Q\) be a conservative intensity matrix on \(S\).

      Since \(Q\) is in \(\linopell\), the operator exponential \(e^{tQ}\) is well defined as an element of \(\linopell\) for all \(t \geq 0\).

      @@ -536,6 +573,113 @@

      The Finite State Case

      +
      +

      From Intensity Matrix to Jump Chain

      +

      We now understand that there is a one-to-one pairing between conservative +intenintensity matrices and UC Markov semigroups.

      +

      These ideas are important from an analytical perspective.

      +

      Now we provide another point of view, more connected to probability.

      +

      This point of view is important for both theory and computation.

      +
      +

      Jump Chain Pairs

      +

      Let us agree to call \((\lambda, K)\) a jump chain pair if \(\lambda\) is a +map from \(S\) to \(\RR_+\) and \(K\) is a Markov matrix on \(S\).

      +

      It is easy to verify that the matrix \(Q\) on \(S\) defined by

      +
      +(53)\[ + Q(x, y) := \lambda(x) (K(x, y) - I(x, y)) +\]
      +

      is an intensity matrix.

      +

      (We saw in an earlier lecture that \(Q\) is the intensity +matrix for the jump chain \((X_t)\) built via Algorithm 6 from jump +chain pair \((\lambda, K)\).)

      +

      As we now show, every intensity matrix admits the decomposition in +(53) for some jump chain pair.

      +
      +
      +

      Jump Chain Decomposition

      +

      Given a intensity matrix \(Q\), set

      +
      +(54)\[ + \lambda(x) := -Q(x, x) + \qquad (x \in S) +\]
      +

      Next we build \(K\), first along the principle diagonal via

      +
      +(55)\[\begin{split} + K(x,x) = + \begin{cases} + 0 & \text{ if } \lambda(x) > 0 + \\ + 1 & \text{ otherwise} + \end{cases} +\end{split}\]
      +

      Thus, if the rate of leaving \(x\) is positive, we set \(K(x,x) = 0\), so that the +embedded jump chain moves away from \(x\) with probability one when the next +jump occurs.

      +

      Otherwise, when \(Q(x,x) = 0\), we stay at \(x\) forever, so \(x\) is an absorbing +state.

      +

      Off the principle diagonal, where \(x \not= y\), we set

      +
      +(56)\[\begin{split} + K(x,y) = + \begin{cases} + \frac{Q(x,y)}{\lambda(x)} & \text{ if } \lambda(x) > 0 + \\ + 0 & \text{ otherwise } + \end{cases} +\end{split}\]
      +

      The exercises below ask you to confirm that, for \(\lambda\) and \(K\) just defined,

      +
        +

      1. \((\lambda, K)\) is a jump chain pair and

        2. +

      2. the intensity matrix \(Q\) satisfies (53).

        3. +
      +

      We call \((\lambda, K)\) the jump chain decomposition of \(Q\).

      +

      We summarize in a lemma.

      +
      +

      Lemma 21

      +
      +

      A matrix \(Q\) on \(S\) is an intensity matrix if and only if there exists a jump +chain pair \((\lambda, K)\) such that (53) holds.

      +
      +
      +
      +

      The Conservative Case

      +

      We know from Example 20 that an intensity matrix \(Q\) is conservative +if and only if \(\lambda\) is bounded.

      +

      Moreover, we saw in Theorem 17 that the pairing between conservative +intensity matrices and UC Markov semigroups is one-to-one.

      +

      This leads to the following result.

      +
      +

      Theorem 22

      +
      +

      On \(S\), there exists a one-to-one correspondence between the following sets of +objects:

      +
        +

      1. The set of all jump chain pairs \((\lambda, K)\) such that \(\lambda\) is bounded.

        2. +

      2. The set of all conservative intensity matrices.

        3. +

      3. The set of all UC Markov semigroups.

        4. +
      +
      +
      +
      +

      Simulation

      +

      In view of the preceding discussion, we have a simple way to simulate a Markov +chain given any conservative intensity matrix \(Q\).

      +

      The steps are

      +
        +

      1. Decompose \(Q\) into a jump chain pair \((\lambda, K)\).

        2. +

      2. Simulate via Algorithm 6.

        3. +
      +

      Recalling our discussion of the Kolmogorov backward equation, we know that +this produces a Markov chain with Markov semigroup +\((P_t)\) where \(P_t = e^{tQ}\) for \(Q\) satisfying (53).

      +

      (Although our argument assumed finite \(S\), the proof goes through when +\(S\) is contably infinite and \(Q\) is conservative with very minor changes.)

      +

      In particular, \((X_t)\) is a continuous time Markov chain with intensity matrix +\(Q\).

      +
      +

      Beyond Bounded Intensity Matrices

      If we do run into an application where an intensity matrix \(Q\) is not @@ -582,6 +726,12 @@

      Exercise 4\(K^m\) representing the \(m\)-th matrix product of \(K\) with itself, we have \(K^m(i, j) = \mathbb 1\{j = i + m\}\) for any \(i, j \in \ZZ_+\).

      +
      +

      Exercise 5

      +

      Let \(Q\) be any intensity matrix on \(S\).

      +

      Prove that the jump chain decomposition of \(Q\) is in fact a jump chain pair.

      +

      Prove that, in addition, this decomposition \((\lambda, K)\) satisfies (53).

      +
    +
    +

    Solution to Exercise 5

    +

    Let \(Q\) be an intensity matrix and let \((\lambda, K)\) be the jump chain +decomposition of \(Q\).

    +

    Nonnegativity of \(\lambda\) is immediate from the definition of an intensity +matrix.

    +

    To see that \(K\) is a Markov matrix we fix \(x \in S\) and suppose first that +\(\lambda(x) > 0\).

    +

    Then

    +
    +\[ + \sum_y K(x, y) + = \sum_{y \not= x} K(x,y) + = \sum_{y \not= x} \frac{Q(x,y)}{\lambda(x)} + = 1 +\]
    +

    If, on the other hand, \(\lambda(x) = 0\), then +\(\sum_y K(x, y) = 1\), is immediate from the definition.

    +

    As \(K\) is nonnegative, we see that \(K\) is a Markov matrix.

    +

    Thus, \((\lambda, K)\) is a valid jump chain pair.

    +

    The proof that \(Q\) and \((\lambda, K)\) satisfy (53) is mechanical and +the details are omitted.

    +

    (Try working case-by-case, with \(\lambda(x) = 0, x=y\), \(\lambda(x) > 0, x=y\), etc.)

    1
    @@ -710,7 +884,7 @@

    Solution to Exercise 4 Semigroups and Generators - A Probabilistic View + Stationarity and Ergodicity