From de0530c3ad399373f28230dc85f9081b01184099 Mon Sep 17 00:00:00 2001 From: Qingpeng Li <43924785+qingpeng9802@users.noreply.github.com> Date: Wed, 26 Jul 2023 14:04:53 +0800 Subject: [PATCH] Add tf32 doc (#6770) about #6754 . ### Types of changes - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [x] Documentation updated, tested `make html` command in the `docs/` folder. --------- Signed-off-by: Qingpeng Li --- docs/images/precision_options.png | Bin 0 -> 42038 bytes docs/source/index.rst | 6 +++++ docs/source/precision_performance.md | 39 +++++++++++++++++++++++++++ 3 files changed, 45 insertions(+) create mode 100644 docs/images/precision_options.png create mode 100644 docs/source/precision_performance.md diff --git a/docs/images/precision_options.png b/docs/images/precision_options.png new file mode 100644 index 0000000000000000000000000000000000000000..269560d80f0b0f1cb9769088fcbf8b030144b63c GIT binary patch literal 42038 zcmbrmWpEtJvNbA3ir?j17_zO>XwCaDFY+ z?df+L?(cAc+qc-yiGMu1ZS+#8>f71POwGC+r&1kN!}Dn10D-6?q3q$>;X?m>6v;pp z0ey%?r3SUb0VoFG|8|fJ#rWHbioW|#Yhd8#zpZfa_W!hs{O3=Po(;-SyA3LsBDth~ zKeG+YKdw~i8t@tME3q5P4mENOm=XkhLI&F_zWH3>wl#-K6vppjh`a{P$m-)TVmlyVMmEao$5OnDv7u(z5+S7i6`mXaj)9tJ`%6%}7MnX6#`|Mj;TqxNTaJvBkx&X^>-jVN$t(v1V=Vs_{ifR; zHX?U1xINLg#nK!U9FQ@rO((#x5oT&%>WP_U44( z>0(8~E_$(AkCOCMJmvY9{bs(EE}|lZ?4PWa^ri#9^{VrLhkw2b{4)?5(+Q^2>GJi%l)0EfkR(Yw zWaWo=u~IG#!ooeHY-&rKyUbzS+pfeHA1#!=+u1{MdBG%cnQ5!+!_6H=IJLxj&l|6< z%vX$b$3-@u*E3@$i;26;gvXCB_>A^)t-vtm6DK9Ku&B0JOM#~)ac;bky=vd_L3i-9 z<`e0}25pd|?#?^Jce?XYF6fy%F>!Br2x{C-iRL5kJbW7|WD+d0qYs2gJh(KN%R(25 z88qlSxIydj$5zCIF78a+SgR|!b`#3)#V3ehvO-f)_vHB)D!0fq$ZexZXIyh+E#}J4 zom9|sVFF#<--CarR?4`D8lx!T>$_994ST4Do_i3uTt&+_CYR~#37M-%vy*#9FaGb? z+@EX}G#j3>^;yMw_h|C^tjL(S2umA26DVh-%{O_s9Ontp_<`w7OSUzJ?lnnlnOJQi zFIKKalskTrwqRQ0Hrr$UHDbPsZ>>h$?tln{MFLL&yGOr_vR z%DXJ32iRI78J}fNSkjzI7KBWYwRm3sz_7EFa~?;klSRJZ+rL zhxeA+Bv0xL9FoBbL9&_tXUh!zO!p`DLcHaKd^sgE>P8(nJ6v21>Awhu*8Y0!h3d7kWec;6{ zEyLEYkYOXzFCBo)>5J(k?#;2nui7LIgokfDA=QE};k5J`wWc!IenMbsvJ7&(G5e`U zdZsx>WH|b-n5~5iKU1grvkqVIP~z}z<2zq91bps@sHU|BkW)xP3zSkeBU@jXs=8~q zwdS^RwtmIj`zk}E@V^Uuc)vp!g4@*|rZ7Kk-;n%zAI1+SM4m8Z$E$32-2Bzv`Y?+& zpryK9zWtz(wLPU_7VmLs;?#J45W_RRqICekC6-UW>~A@#l4!9oCnzY-4E$`b&Nh3R zvD2WCCf6Hn?R9542L1fZ|y9t&Zt*N0-?9Co^0^&(T$tQ2vsfrl5qGIuavdpR~1*=D{*G zjB705T8J%hf?8#{&KhMwfaLg2l#(KcktUC=mED_}P%LDgMpc+wC)sD(SN|=tm4GW& zWuFxF_jeBQ_{}<`qpe%i2mPfO&3xoA2Qw})_}vH9I_q(&G3m^be0E8TUtS|fndKd! z%$pv_*>hr=svM zyi#?78-^+nlv^CoUAF|2^*`3dMt)Gz@!~g11CAf|m>mq^x$~~IL!-j-)-Q(+_*#e; z4%QxTgF@Vl4&@=J2x@^d=Bj|P`yU3@$suP8IqV@AI7IR$NksZNQ!X%>6;Dq@#9ogk z*ru&4ntfprF%m$}<;G{%O^?qc61Bm@B)B5GeoIhZ6>oP|T=d)gUd%C9ilyu3?{HDBl2qx5km%jcO;^3n zEgQy?t2xIrhB$YDI;&LENPkRh+dz0QtcIVO2ax!q@F%2!4(D% zKwm1!WZPcv9g79`PNa(C604l0IdsvWt#87l6&}n#|MeU{E3i~_Nw3&0h%Px62RGpNSQ^KKLxK&wA%WgKMYAX98?*0;XmK2R=DwaP(gY=?X`P#Kq zO%pDobu-5H6&$X{-q86=dk5i?j*V2(-sBV?`Cop>Kt@<_Rd~8Yy z0l|enJw;B%=U2@)APg4eu``|L(v*@7FSQhS{jB>k{TP#&lJvuCUIvMUI#}W0FV`eF z9(QTsUYCRB>@Ia59ko2DuYf@f-O?ZFZ&|a=?tu{SVev_gFj@sYZQUc^Ne0*Z+%ob!1f~bdSE(o--gp{mq zrOVM=>F1h~=b6;&9{CHu2Gj%qrgVyAJ)fj0i4#j;tP`)eh`h!(>h;vxxP^L?OV{6L zlndMBL+ghM>1~-SKDXG7;vWE&h_Hxg7=yQp7$%c+9oj?)37_(tMQI?1J8~GdG^*x@ za&t6j$5u;ztZP<1zJ=$x5se*=D$u0@O`&HK2{s?)O8=1H@1koXE4}kMM(kcgV-Bhn zy*>{tNCzsF3*}uOqm*Mc)9XClHUXreXl5#>l{V9}E8QT%cdI7UH`|T~DJOm!T08q) z_m_DMD%A+PpXP~&3tOpbM|o&!HRZ|zpD@LgI;2N88^_8u?iHQzlh9}0uuY!2=qM@p zo+1(L3HPWAtpyJnY3Z?2@(9s)8=6E-CSb>tzFq-Eq))1#$C4@)6|IsIv+|RMi9OF_ zvk8KmH6nQ%{VTh#7s_rl z6w?2k_1bP6a}{J%N1%)DxM9%0%VaB&Q_Vy@QfvAU&^y5k&Z`}wTk2@sjof%xnq; zM1O6qHc=2qZjXh(>Kso=FspaJr~ZP6CihJLFh_Snley98VTi60%6Yo1t~k;VL-qT$ z$wpIz((7aV2N%{)u2pd_5su(JZy!<2s60cK@vP5p-n+M^c}97x+l3u1xZE%v`Y)fZ z_?;P%_WG^TjWNM8qjjwW_B)#o>rGNZV(SC@=tvZ!Hnen$wK~PW&8gp>{w7R7{w+2G z^!MTL@kOBLhsMOzz_E0-5Zl%^BKT5t1j(hb2#aAjYTFZi=%R z8@#^2?sgnq7~+-8?~BtLD{`=UVXVe;l3|&U=?TLoF@#~oo~K@JvED*SsNQ0Wp%{ZtjL&jPX?iI%)56O z`|F@{w9ORG`a2IQBbQ9f)E-aa+?LzKS+!@S5OFk0c z#ze!A;9oAO#uR~jjI;&j%sEh{6Pp8h=SE(k!y(L=@kE|5$`*H@ydT*Gi06os%1oPJ zGB-o@s`*bRU@Vgm0xn1VRz-wK1eixqYyX_d@uft z3MOyJ!kdX5piS;A{P#g|Uuwv}MyUCnkyY*$($gOLK%QGdaG~@!Rm0(HZ@}Yy`xY&O zi(z$HDq4@JjPbzWl6}9gO_50f41{viNBCR$j8Uyv>}mmD^SU$r_(Mlu8qp2j$2)g} z?+#a*D8Wy3)9w%At@o}hODnegs@9)2QFBTfX23hot-7j`a7D?TSgnL;dj=x(3AMP} zj%#_Fwa)bB&9@|rtuIpzw#e^VBymFrDf(ZJA|<6GzS1C+Aepx5yV2t~%L@z@Q_Rq^ zesej;leVox$9r-OzE%6)K#}kZdH>9UcVE8C!(!e(%kH7Z-j&V%d#sLy*@AzPs>_+e zfc`tN1uGTR%tO=giOa%W#p|;PlkY3W;zZ`)RXKB!W5B&cmt;Qq7eG>h-3ZWarOgEP zQb6}%>9fajqcb~l2x1?8d@O2^Ue;wiD)m4Zl&Tya`AoDPqYkeE^F;sY3eu*%X7R zHgb2?%TncTm;Id9`^HPhRxQ7R{#rWRgI;p=M!=G7-uRz%_ubTXa}1-Q1kM2Ht=Ey~ zyt0osH}*mFVs@oUR&*j9i2}741W9?D$t2gUX?I9ojM63aa4Y#a7Z`PGrTlR7fj?GN z1>Xy>GR2X4Nf*=(peAMl2@2ou8M+w(yj?tRArIS2F)6i=>1Hj1ibIv;Z5U?kKj%gl z#k!RjPQd?!mRy0X#LFDdYX12t2}W=k2in(-=u5H0+~BDvf8 z?-PBLBsYsF?DnU7$FA?3pR7|NVHC?bH6xkIzA@$X^@T~>jBeiqU!O&nGCUnxR57gg zEF8H|^yLMdS~k76#@5U*UvG~Dj|e@Qam%^U;cPvl65I>)rPMe7dAo6CIWa*30RU!D z?3iQ65d&BD2g9$Iw-qLW^vKU_&VQ1Y$`}2LW zn(4Cjw|xDIea1v?BJq~BZAT98s^|~+j^^1i7-}smv=s|l%(Bv|*@5S}_%#l@M?LeDX@$pCL6BX!Dt59O)KTZM^L~(Tfp3X^}hLBW^yFP$xru?U-h_e4{SZJ-VCx%5tOJ7G!ysrGmFK} z9$Fpx2nH>z9JGLukxME3_*$$!3fmr7GSNWD8>|t!y?0ey_jvSlUb6;-IDHgNb2^qY zjXa?-D@^k(M>5rZAdue5kYdnE4_Sg77@9Tf8C-YToBU*xsp`0+zF=cwGs#)FLtYlO$ZHg8 zF^${#K_X_-MYc4e6E8VhExf3>=I^DG?}T-ha>;`_K3HlhEc*!xSbdRvmI$`xH!k1fUO)8@?-1P*VbK zYOdY{t@d>A2~KvwSsQC}8oIZg8M zNhklA7(`(!pXh~-@$TKjaTPsdG}@%z!+m(-xb}zj)TZ;{hSu_=)wuokI^pVNcO{u8 z3mA)V&}#dGW9rMFD)^ANS9IvVLQ#l;^UasP|Am8Edi8Xg#z%pYr`^H6Ty+^8KNjD+ zgU=7u#*^0lB^-EmJ5bC;HdY~U-#VcyCMJxgW+NyJmWiAIh%8yg{ek@k*x<#a49u^K z49{aLg9I1(IcCsf836$Bq6c7$_;8Coz0f#9u?7vsr4a`ZLw7GpSxBK_xuE2tazd&P zQ=sCQlvldLx6zM|()XMwp%JHx8ou^+(*jWL!<7+a%7HI$=8f7DO7;Wd5(e`n4+`yA zIb{^tujqVq0jUKI!ONDmC7qa8qqzmE0kmv>iGjE$U#S7|p&x$L84FvTW1i&|-(1h` zoXq1R#fW;cx`Q6{TT+j@HHCL$Uw-}BlyQX6$H#Q)8);PD40mm|VVBD3ES z@nII2?3#Li3R-cBQAEJhkJ${m_d$q#+h2ocw!Tw|jL6AV+LGfiz+dyi8H0RU9UhPi z&&Is2XHwD)P@i*AE7tow6~2sk=_A^-T&Tk|$PH~fCUjZ==rfUpO4E`K;ym|U$wpKD)||HRLv4IhA+qk%OVAsjb#S1X zKb*Xr3{KuQTc||GTdIckqJ~p>6cW>Awz}^AL$fi+n+XSeLbQ;xcsfVmo6AM!x8LOe z7+N~OKU&A5D?GFprir!LbQPiW7h->0_z0OowKEus;1DTr`2VUTp&x>?tfC;Dz2oY!-t0)N?s^WNOy|acgNhwu`=NytAkciD)(dp6d02t$7{QO% zghs0+Wp(x7r87!Q1T;7(%XAV|IxNdZRgNWYcX#*m_0|M_&wE8pO-Tm_Hm-Cz|BMoD zLqkJJIl02TJpYUgLYJ$}DDiv%EiHVeyeS@9NPMx%AYO4SB`qQQu)I1 zoyYF(uH}`sk&#hYXlTz5E(~n!;89}`faNWWJuaHDRWFU#nT$Vzj%g(o6$Rh>gCp@d z!-t{>#=EfY#Az_|N=iaWd|%@@trl~Z&hWT?;6lQD>M^X91c!&W-<>#c;?=4%hF@#8 zEd~v1c2=1Xc^J*R;1diaS9;XQh$I6x;>^s9sHo`tyRV1{1ZW~m?`F8!)5C4fXOF;P z5?VSsGEUAUrF`+3HfJjf2lP8>B_*^3Y6a8DG)|BqVUX|#di)@3SAHme21!hx9j}z% zd<6BXo9~A&TvQYlg|xKrprN4?lamdb*FfR=_~rxQ2PTk>5+xqcYamp;WW<(gY4rzW zxzLCRaxSjqwVZARYAs(Xq28Oel=Zb=qg#Yo$H9r+;M3<^`|Mm!tib{e&N8c5F<5?kk-*adM^~wj?s;rC- z0THow?PB@DrL4UC?_P1?vb4Bes!9i1UyshLk`fxwO~Rt1sX(Ived+7#tI@7({9t}l zdH?A5l9xwLppr8@Tde#wX*$LK@C^o;Fm%*7{x{?4Wb5S74Xfj!?BPUez}iKT_sjh^ zFW?{r1;vOx6F?qh9Z&gSo91-T5HjSclai7?XUd01M1b70xS=8O`r2V^d_2ajQofk{ z%NN4(1!*_8&)>g)7Znpj!@y7>c+AVoi^f0Sr||T6un-mo|2v9}HfP)@anJ}xYQFcg z;72Ln=jmm(gKpoS!Q*Jx`mMocBW)&MA|XDWY`7JqNltAhh*xo$4AID%#r8 zK0X2m2M3LuZhF-?+0z>B=YHCJnerie)x!oX^IXHk=4nx3VWn?teg_TOl^IS7wej)j zAj)oZwopc?RMqKfGc)&V2s#D^xdLs3NBzjI2hJK~!8E z9TW_D)hQzH=^%@Bf4uY6n@;AnwWWcuzq!=4ROtoQ70*Y2|&Zzjm*gvb)sBlNVixfKg9u`_21c2;ZRV9L2-zMC8ny12||XyMu-6}2Hk}J8Wn7J5 zUs+kXKwN&Fz)$EW2(@nJ6a;H+w*+?g_V73@F)vmd6G%aO4jix>rd^2CZUmziKAu5P{dP%>={QcFQeYzFDwl#5 zT(eRgB!*x|1c7v=ufY;wQBi{iEIBjgmS!!AbbvpT!S63H*GFM!0JcJUl5UCk_xmA0HpD zQfUYDDl#*HQBqRMX7VN#%B18iSe|Tk3-$*ik&}^$Ad@NCf_BHlsmyUu(Esu1Yz%-H z9vqa4O#r1Re8vC)?l&eJn`VNP@u2a8ypE2H`1ttMY!KRb3Fuo}>$BiO^_X)Q)@rh( z%7PH>2Nyx5PIHmt(Uh5!*$Lzv&SVBpg2{L?+9#Nk)kb<8JvV@ol9I#kP>;PHj8zcV zT>y$k0qy=1 zgq)sQ($Uk8PES+gelp}rKUr_RN%d|nQ$`~qniRv)6$RPiZ?wx7--L!pfVS+^Nz>h<;0FCVN9s1A~Kc z$;niFdh;?q-6ZWB(UYbLwR?YaHm0}~-rHhY*T$3*e@p2?1vmM`NjhHQ)mp>KdzvW= z=E7lp%Y()v6@#Jq1>}DuhWv%4zZoL`dp1dixA7_?V4yn#A(JW9%@Nfj` zh5UrjTIvD&{8Q;6ezml?49-3{5Wci-_d+F8iEgKoH4T0STrCJavG7AJ7!_V{9elki znm1a9MLG5J71_HGwg5N8pE>7dpe&e-Ft2^F0yt5rG=6IvvF5;37 z^*rGTQkZrgoY($5zLn$~9nV{Xb@Flaev6UTw?3E3j)Xml`wUjTD8jOzH(wjLurgiO z;7031>2!;8`oh+7@Y2vUNcgJzTP|o%Ve#BEqA;j?d;R0AVp9xxq`kG=$FQp{A2|0$x?QBNcA?r*55 zehl6k?kgVkHDfTXE4Rt~2B} ztYzMOn#x@lG}Uy58<(v6``VUcIz$rniDoBgMn)G?FIs0bC7q8+8dp^OCs@zLNd0P@ zaZ>IoKswWSGKqZn^QzJV)Vo)@Rp|ICNHuWdDCiVehJ3(*bPsZSHkdvt4a^l4%`%+Y zxKI)}uvz$8Q&IZ+R$!W)%RDLU<6_UH1Syy~I_ZwRqz@E?RxI}g4RhUBpv?B;tHEQ0 z7o2k2*Bt`IHe5_0ct2Ozz!-m+4{-JVrkLI%h%=Vom*8b4f}q5uJmPtTlSvMnrKsSU zVDb;#jp+H2H#s|E6ex(}ikSxUxh#tG)3fD_K0aC2afny)8wkuP3M02Q{(7bJ?loL=3e z{_BPEr}s&T8Onj{Ax6kBmW-VZy+$btZMHw|OZ5&{pPSsgN>tZ=r>(J4G$O%M@e{;& zdc*GkFHH5H2(KZ*5P^8SvB{fJZZa^>*Vd2GZc!l!N6K`_s&f{~VUhO!dBqc5p_vDs zrwnWHE;B?dP_A`uS92Bg)55}o(RFwwc4&&2PqXu`e$KIP3^g!iTz+{%2ikB~aEq4) zV_;iVX{4&R_$4AQ=|YX$YDfpPu}x6Q%I>U;`pf`YeG4)AfoCxQaOr0ww;a*P}kkAQmlk8Z~v`j@fIx z9}*+FUe)5FyW+&!efuKp+rnmWHAnU&f$4Q0xx)2a8qq6@J-(~{Ow6x&yM|{sCXYWz z%1p|0`8tSXcf!}$zC)AE(I$)0UtE4n_U)XB{emYI%Ad-|;~J6qcID~zm<%WgJSkm% z;N#YJ;t{Nx+Ihetu+y7MfeYA~VEUuTBi~IF#gtkwvUtw)O-{&Yz*JH+AJgdJeW6I2 zc{3+ns3y%)A4q$AkS|ubm@^cU2c>9P#swF6er>{|LxZLGH2A%{JTR67lZ57-x7&3A zULG7yU6=-L^P3`s@6Oi%RHsF+VCE0i^(=qs=I$XTNo%A!WnDAxpueYjb$q1fN7dDA zPsk<4^r3lYc14$?xTAZpot#mPLcx^iQ@g ze&+oNVNLU1QT@9*bMF*>TcrXO>(aQ5xXx^40DJNYMze|XAVW5~x- z#hiO6m%8ec@fpZ_(V?qOwgsd|l-Go|&~~q24qRz|Aq#)NBb#9068zLwyII|*@8u>y zILh>QW`Z3i*}g!On)df)P__9floa?G1aWV_o>n)&$_OMVT~OsGYrVyUW%Rn!1=AxWDJ)|pmY z5kAXokrJdVj4nToyY8ldg{|3 zaWOu6QZ!Ies~prTmOO@|jC1-2L&ZM14fv6ZLy@glfU4s)sDRT)*ID|m%!%Ds1j+M} z&@Li)?uC$QSBTUw*Ewbr()()#EyXs!mJ%y-ypqKc4@=9qG!vncv#M?XZ6GemQzj; z;a}G6?peO>bD}O`&}I8wBT48_G(ueq1h$~XK2ki6lRm8NJ%rtu3BSW~qHvW|pmGr^ zXP2Kiro7R-Uf5U-#y;RK<9wY^3~hyCn+~Ya0K7o{;rmy=m?)Y(4waWBAFC+E*sT}{ zB*jG_iosy1sQ>M~v5yIqIo3m_8CQMS`<@7uYTu zRmRq8i(7hgiY0-ah5E7jPxNPquyfouCfWr8yiv=d7|X&ZTiN^}ff~pWKKZ`kh99)fG}jo93amNyniX ziNpc{J5;n}6=|-D4*~w7ZzDEaf+2}4BLT0z2md7hwc8rHunAu6GGQ873DuE#fFocw zCCBSdia1jJ_b?7Im|m3>`zGW^=v{B{Eb&o;XpF6v2)1*cFrl$l&v=2liB;4J2|Bcbhn zOlYLBxn-1O&&1E-n#+1kKgV(~bRJD^rAUHSn}2O2rSHoqURm^T;VKV< z5#)Li;p!|sIq>ynLQ)9#A)&59OjU9!q>Br})lT*TDO=WuKX4nd7%HvORMfcOy32~L zGOcJ6r=QZ-a$v@TP}eZdeSf;5;Jyf*zU6GfV98jS{`x1aUIM}6=IQ&n%(=M_bhG|q zR-zsM{rSPf3p+{-O2tfEVEQN6qw7x|BYL-t3f-?nI?|0!ZLg-6t@dzXhtIc3>Lo4{ z#=TkMu^!|XY=Xry7;wQ5sv5S&U#Hp`=(L^v(>RELtj!K;6O}I?YZ1c;O^;%S_5N&0J1v$ZzAmv`{S+7 z`tKW@sM3KoKBzai$4;E=l#htHuk<56&=AzWuzHt?R*Bsd$-w-)N;1IRnZMp`Rff%h zj7=u-b#)_0IY1eD>Y(NbuVk6@1(Xbox~o}-?b8dp<=Zd6a8;49;{V9PxZuvG*+7Qa z&6_0*IbG;3?Pi@HobSd?(h0UiwIw)E^Pz2i9d#zNWfLSAHVGd|zP z?(XLpHe3v$%vbv=3BT=fMX}V1=6d|OpZ-zf&+z@4W@YKx?M465IAE(Qc&w)vOhbz& zRlnJ~XHU<0|0Qs2IrD)=P!*TPiCDnX25Tj#nyt};YX*h@UnEfEAdzoLX`3!ys(A_~ z*MY_ZTRM6Mg;N#moer)tB;zHgfz{l~)S_1(VrpTVS#avbOn5V2QN5W zDA_*GXC`jBgL{b|HRlUM8q-NK27@hG{gIuGHz~ZY7rFL8_9GxPF z@uL4C_|G5XHe}wIXd<>yUgIBVeBeehHf=ZWMq8xi&IK-(0ZJ}es3P+^bYgLMX+fwG zZI9y@vU>V1v*Y?pd)m7{+6%Vj6L7KC%ONYJ$3)<2KYVomFRE8|^~Fqopo$Qx#mgzo zEhl8(000aDH1*yU=;K@G!|JBQcK8pn7qKUjt`}dN%v}b}S*-6u{#jp9;wDIR~3y24Z$Y`g?UNoxfK~u@p z?K^Q&*42Hh*3$=xKvlZ!Aae{14=-`)BmSfG4Qzp^a0yFGIs$H+{Qwk!$0d;Fii0zp zBOEX?GP0eb@AG}GNDeBX*Y}P$LW3k3ru^R#{-p(XWt~V{q<&(EWQ^|DG5`ivoR7-X6Oxm~Nl{7J z*p&T2Za&coNU~@SL^VqQ5#}HU6{H}54E6~o05kl*u@(4eCYk;JsXE)uCPm?b1IS~L zcxo^k0E-AadX1Q|77O_Lb(S^Xj>ufF+oB)ar7Ls4rvPrP(fsmhLOAK0>)KnO8ntBt z#4nD2w<0z1Zmhm;Iy5or_ZI0sJvPGNbDRjJFQ$rW-VW>PxNd46R;&tHtoS8yqGM+Z zA5~3C#ze=$lKwj}Gen%TqON{#+o_y+l^X&|Oc8KZOHNoar6HuN*&9JTujesuuhyFF zP>UK44~&#ffru_f?HbGL&zIg$5?qHVIqFuD(?6z5+Ov1Ntb)BxLHmQx=FEeQ? zw%l0;knnEc6gF4_&BtpM+z-xtcP`iT$=Z8Dv;RowBGO>n(8yh=Simy`U&=_Ybbj>TQfBuYA1^(HVl%;7{_EpMme| zjX%~unxhD~>ue^a_1Zf44f5UBroKIHyCqw%kIYY#093o;KDvPLGW7`X;L)#0UhGC_ zenm%r4z9+vJ|7uruPk1VI!^!DYs=fKvSxu&b1`?^d3$YgBBxh>ov80m(BcI0>3QEc zHmSoX=t$A#l1_z_m327GgVrr;9)y+h7)VVm96#(iKMoieDZZ71CnfDi4mFs^1)F}v zM-y?Xebd&(EsaW_BNKC-ca=SMCQ^7PdvbMN7t8qQ_3UWu%0Nhgg|4A|*iUNS0x&O! zGfk59#EtxPxp;Cvuyi?6%bwRxC)+chZ-B-_V?2J;tKr2e!NN**3%G@T!_`ok6dvp0~@Fc9{vME7r&%THfE9$6hB!vN}!C&S&bZ z4{OvGg)4*8`q5?<>N)rPV|Djvo>RV$_Jfw#3LaS$)9PJM?-6*{lb& zJ!*cop4@iNco@YkV3<+}t*~j&PZZy$U`fQKKrmfmv1j#8-T}|O{zMb=#SS2)4Tcr2 zF4KFN`D3FdDNhLAv@$w8UZm*=b4KvFr@g^=0IUCG{R@WmO#~kBaQI8JJBrQ&W~_2+ z2+;y~aF!!-fM*k6YBpG@zssO6o4yuDO;KBOVqS_KZm{92PDhl5iYvH*Ty$T%9iW?8 zdyw%f-*@DMJ+Xg;8X)ucD9c`?nC^k!Mz>**J%r;uu5B={Kv? zxoYKLK|j4w1iNmyx^4c<8r|MS60f5t9PTgXj`o~AP9I+~yzK(0Z0rndmNPimb7Yp6ejXK!vDb)(qS{A_9U8t*C%*!&EdYlqn8&HYM4}{cmX{o|gJJXPSs|=nFHyawU z*v*8A^>lGrM`ubLQtBDEhg;3C^gEiLVvF`eM`(Uvlg~1OWF(N3WIu!uqfls;Egz`x zP2#&0bZ;y~3EY!zMH&$A%Ik$b?Us1`;kR5vBwOa~i+EA9!SXTv37T5Z<(i(+Y`X35 zaw~fA#%P1BYZAA+NvRe--J!MYFXGLS2m&4N@|)`}@wQdU39AQWmn~jo%(BmcXVd9{ z!bGI|OLIocliwVLsTtB!a=}U=ZT9z$t_r0la*HhjIOEOh&wT-bp!9N)3}J^Q24gK3 zG^QS%j+le`C1qiB#&nAOHGLDc)W$%dtykX|1pCo9N5QY!TEAZhrIl`(IbyTksZ5q zHu#S83f+J3uezQ{Gk1|R+1q{#&W&Wo*mzK^v5(Xun#fL^T5I^UHfDQnZ zoc5q9@2@j30$wjXMdu|{Lzcn^4GEn_Bpc#eePSEwd2?XzhDBW;ymO0KkmBkJKB=6SE?1Sde6$kmsbFeKXHt4qYmyHVpIyHZcil;x zg4@T++OIoj2r6f=5E-$U^v(UJGd@x*P<1)j=mA0JA9bmf5rt;1+)Tp9VJ=7V4)A{kj!!IP@L+ zEki=y{6s1Ha*pJ|jgaDiE|L-Z7ncVFK}kdFw*<@^A7obLuYqe7=BmOFpiGE3q}J_O z0Ecv=G!JO&4gb3V4+!G!is<3+oTc21_t7%w>ql;(&{|v^3wm&Dta6h|S0_+U{l9hp zr}V8m|GWs`f4m;x|4apbbdCFa$dkAmd;OApUYmV;4Vl(0*FvigdeVZ|=#^wNFQA$7 zV#N{eb5=(FD;@9NY9{R&uO5ac=n3$85!*NED%%yGwN*oi!Il&-Ot}@AD#Pq1#)6JL z|I5n={yW6~DAE7p4F>;tufcx@{`YRaQF(!IL5Uu9j0m}kaAI+9bf5!idN;>~`p8u- z4R=Hy-PUMl>!`T2Ht(SjUVHsDkoZg_U)d;3nu?qgKWR&T>i(OE5_|!bXf7Xv=`)eo z`KnGJcMdfZe^Q{Nj2yZeCd2}=w6p`Q^D`U_cxM)z&Zk;?R?XXxP9>Va>Q#9qEpRLs z=3G?VD5^dq^suO!T>b)M%#KUCa0HnWrxy4ikw!et@!&RJ1sVzq z3C5d^s}reG8afiaLXv)0~^YOPzp>vjXn(qtS8l17Y1 zeuLbut6Tw<;CqY+A%SFd>wT6+2{zlN13U2z|3-+=ussDCz=q8%Xu~}ae-q@u0HIqD z4PwGv-okcS<=L-;FhDH@wtEKS)O%w)vc-{uRc4+)f5Su|ezjR}u_*Io>q@g52_aH4^V+MzS)b>732k8x zqxm%axwOS-b?mBcU%gTrI%K-~KsXvHM2h1sv~kY;oyUXUL*J8QG6?x<2EHxPsOI^- z*XxqebDZ<$cMnC&Z zF=9Gnxfb_GNK#B=@cG-@J(lmwu>$Qz1D+~%b({{P@lmR-?=Ty}X=3cPX@YTn@`B2y z;dt_8lGCdx{T6!+Pi}GH)c?lz9 zOI$9Kbp4;>VhIR(LQv(&;EkSz;mOKNZdT=78g0$lXc#DQCk zX7eexeT5#U#v z>)$R9+xL~zI1|{-*63m++glSwkUCnO)?e^$^ZyL{IFu^QM`>569mRe_SS_ZtU@Dw@ zL#|mbB>r*ge-IRrw|gk2bA58xvSwCCTeDKj#gdVFi<-aA5&ZfdOx^*S786-H_i%5qmO ztb+g`+Dr?(*Ee-%;;TYsq;w+|qx~858O-I_ScpzBT#6C1PJ^~z3)V~ERtx|OBo(T7 z)brQ14f_e#A4m0y89ccoHi?etj5tf1fJ@{36yzu85pH2z0`-`%#J*rL7JjpZ)(MCf zBB0l=o}tEO)_lLz=Jk~1q1Dj@7BUk!Yz<{rJm$Ai)kgmb*Wll-HKJoW69BbFFgG@?NulR+oS1W- zpi%L36+QSu0b5KL?%1#Y+U1Pt%bBLZf_w9Xk;~;SV;tE-V{x?*tMw=B?ymd9UBCO) z`}<(OR(*oA^qF@|7cXW@r+3tUc*u;zWjAwRVXkd{v9*p%p&1uon&&i7JLF{{u&1(s!F`EW} z7$G&^_nmBCE;p8@Ex+J&G`y=>IzFTodJuyP@7`?o1Tcw2!+Y)0DV@n*@lr!stR;x; zF)Q%3fv z@Qd40yomTl_wqjm^0Zhqo3G6~e~1-o1ff=s9L}3WN*{DqiUP}%Z|ui&XVe*GoZJQev>mM8S_oBPl{{KDcq-OCAVZGabd zYX&{<}s^vc$#5C;KqRpAU|{y-s>1iD&c7D4EC&gFT#q0;WeN3ul~= z@cVCk!g3cs>%1O*cs!dtemnT+4wQLmD@ywD?dmzldS07_AVP0)5LsVg?GK|aWPtNl z0O9VVTU93So`yZ{N>U^zDK$n(EAFo=ek)#qbMVcVN@OB_UThUU7^H71(8?3m?w5p9 z`CeOlFcr&M$KSY|@uQ6B8m!|9b|03~l~nmd?#8tdm!Y^3ipmDhx&eTN92;;y(1=0G z{XyNJO74Q|zX4#J=<7e#EeMkPz4=773Z0=;=l$eVET`d9dg?EYD7VzmWMwFpbe&QX z$v~ab=7+5AMJY;D+LRb<70_|xaHmnf#_yENJX11ImlVxHGsztX3{2cvEtMQ7WYSr#*{J4<<-TzpLYI!O1&*W* z*eUc)nP8bMhG&kCV**h~N)3D8de?FWs~!4we#`jG{*V_-kS-=?CW#v#kIkABnJZ<5 zXE5A3z*=tN%Rcvkzjl`y@CAlBEu$nQd4#y$!cY!f8f_PmWpn$!*m;WBZ}sVgubx{= znQP-?@Hia2PISCIY;frMOC^Z3#uP-*(Hu6JKD!6Qq0ky1JoeR}cY+R;N)$DTC)?LF z9m@2GY9j{1^-u->fyy+^s~46HTg)0SXF^_j)ueB0xB+kMl1zf)h?rN#IEUM zE5w`4$%2Wmxz=jC|v@H@Up`GEMIHnUs>*@7A+rv{LH(fQhMc{s1N}7)HVyPw4lO z#C$Gt;CD|2Uwbr>sCtXzZ>fVFKQeBm(rUK(L@!>?)YSr@Kl#UY03bg8i z;s-oF44^Pqu>;DO9>fHy{({V-2mUP;8wss0REErBP*VieB?8tYFUd$90*? zMm!u)=V22-eYOCoHq)pQ`Qwm3%%adc(pasWQp_I=RI*TKF;+nVqhBr;WUYTRP&uWc zeG_^@#hOYRmte2YC7hr8#|N#8S%*yB58h(D(aMk%m0QCIv>r_IZP3F4mqh$O_<^ae z$_&RM(rOC%kwO&v*TQ{8pm3#y=-{>^B>7*j5DF#apFQk=5YA~EcH z#d5eIiiO&9;cG)RLi*+M1Pf?g^3ASp+Qds37hdZghgjlbH`R<=<>Wj!=>#`27xp$t z@Q**|ZfFRVlF?B=P0O1gwP}v!vcBhlh&@+5xlNku-fvr}`P|o4Q|n52-o?dQKH#43 z)fM`T42M3q*7sVXrVgOxZ@R8Zj(1wpzAQFN_9l;X>QztX3w!n7b1cydws=5tsr?WI ztP5S17>QcfSgPJ%1qB?+{_+Qjws4wH4&nwM$0^|K_tD(qP@p#i5qWcZsjU#S1G8hb zOeTKgjy#8(#8K@1K(`fSSFM4T6Or@+tIjM~BpuSyKzn>-@CNPm|MS(~>tGwuitog; zmlqI=$s|F)RU)Dg{Cv97@uZ^fRY#tw5HI90uN_a4vt&bk`i090u2G-(6{MZA`jta z<4vYOA;PFuSH@*;j(DMp#M}G}GQtPVu*6dlBY)&&_9a9>QgJ8&acP?LXXQCBX3m|w)ZImMc`5<$Yi?fm%CX>12p_T_{n)#V>qTcPsq2Rk4OKDRx zYV2_?_Tu?aOJ`89-ncKJAT&P<@B8jXx4TKeCI^1EBc9#N`uzp%#Yw*7 z{L)$GJ@pdb2Nlolnd-7gGm$r6n*mQ?`JO+TUkYvZ7z5tz70&mgtzziwq!E#y(z?Rj zGYt(8O);8%uCQLkJafzL=J`3nLA2}#Ju*edINB0=12AjDmTdO!XI?4ysU0~_X%ye- zCjv_CC_yY|6ou>uG)z|>QQp=}8Y$b?18xrUI2bvwMGnO;=cs|=Q}|vBpHXJ%b@KbN zGPP|%=-@jOQ$~XF<^uG=9gG|j2RXeUX#=#~6cgR>ueA~GSm(&DxQ;c!KMhDcsV~Iw z74Hn{$%N+jrz0~RH|c4=f1EpaR?yb{gnoUM@6dc4Lr>%3 z6?N^1AA@=%rrnYD&uocflcBg7yyUp)k93tk?x{lksih>SFQgvj<5Dbk|fjf1&vXC+M5GWFIBS?>xo4hEa+pH}KG)u^QkrOqgE$@Hal0OIPY_S;YC2FTB5 z$;;=|B4JsMCRn)N_E|YN;r3FNk=)!e!_x( zzNxWbjEx;~NeR+6(Lp?j($Ho)+7*+1@#R%9cQ74INmyyOCs@K})Sx?!sI2a%0(J2ep73BhrETu( zOrMU5`g>z`$DD<;)D{yf9FLochGiVPI3IJ+V)5h6?UKXQ0)vd#ZEsh7c)+g!C^o*E zC|La>Sb6_W7cpZ#+;(WFUgaL3B3bh zc1Vk7R?XhH&D;-)jK}ZxhgDG_%k_q+djfg=82Meg zYlq-)%L&eWQ8s;#q6I*dUN{>Ki0F7FvSqYZk!sr-tB=h-PV)VtwZ?y-IzPavcdMDz z5-SI}HB0tP6<(?<(l<|se)$nu*}uXF=Tcr8Ek^*!jTo_>divF7B$mg%ES9)EO>mM0 z_=WX>4dkZsfhjrz4Y+4oWrl1r1+KiR9|${x#OGR){$M#W1N!m6G6?w$cGZ%Wlac$L(5u8PNNJTJSKxde4st!aifVv0F!Qu% z<>vcSTi37gz&Q2<({qd_uMHG6c_7Il+8*^^TCGI z26fu+xb&-u_+ejOC;Fv_F|vb1$I|VV%l$Eqyn}2UooRB{y^)B^QjNj-WX*h(fxv`* zD{Lrflji9X6C$9${9JDu}AxCfv~n)*~)|l%emfKMd=kP-rP-(E5+BZT=_h zR5y2!u?58YrEf}hgn8c8s7k(oB8%V^4qez?c`N1Qf-UVUfy6IlRmDF~X$!;YGh{V1 z$0MQ-oze20-$&4XkyIH&VvK5_*x7}}aU<)!Z%9)Pi>2CN;63(;d#gHVhr10GAr?~` z)_rOu8G22C!(^c|vO#F%Qi!-Qu_04?I&&RQwn(#ivIqL%iL!y};U~JCEOI1*n{9rBvi-fm*RA5&^Dk+zO_ct*wao z#s>=N>d3ao`EHfRNCV^P=V1kF6M2w1MTcIPJX;hKPV;#63cbT9d+C&}q`VnSG{^0z ziV0`HA4&1NfG@}a?SXI{PabuqBf)H{Uh*Q0iZ(C_)>60Z+{B`8{!FP6=Z0cRnp|%9 zd%ZwzD=J`^gYTEyA*--Y=>v&I#WaqJ)e1dq69yOtM#@T60u1R79SfF=`|E1Dhq>y< z7G5@UG9}D&EobZ50wt?yYf?s=6L%$;0`+6!qv5)@1?ZqV(JbbM6FP{Hg`D_2#ll1a z1IkeG8i|_O9P(FcqDl`?_4?-8+gH&@OXwhBN7~nzOC0dD<}3=;)VYQ-UzF-na|GUE zGEfqd3)itDu4uqTZ)O zmlfSMYcxP0TP$x-1r07lh*yogI^zK&Usg9r1xx-mPvkh!|I9n#mpEU?>p=EVN_m;* zd0jWxg-)x+(^DCq;$16c=L3N_T1fbt9E;~+(~5-s5`)3bXh|yPy<02IA9KL(j`6&{ z+=4|>Uu!5}(K+u^a*~jTa!tM{3WMx^q_QM-tgYYRCQzt;v-uUnxP&2e$|7VLYQ(n##} zm!O(iOQuwOi8HY`7hGi%t@^#9#+mfH&*x^I?T6l~&0Q>!Ty+PN5>PBqF=GRKH81Nl zphR5SW=@VmV%+xE3I5DQ*&-T0Cw|t7w1#`A*yhx4YJ+3_VrU675^i{mNHrzw$*lSY zx3Ko{fC$>klz0Q8$sN)_1=JXhVu?7CE^|nuqF7W1JhbX7E%)AOe8cCzPSKW;6^Hir zu=hNxE94Ib)esn;3q!K^aY6mzCIRDkv42|zXk@kgVFQRC(xH9A_ZojcFmd1aj7pfo@%pvGw)QS*{TlV6!;b7)`)W-_^X8Ow zarJ$u6?Z~IWvkFfH&@@Uka*3aXB3iyS3+q3-#@P@7c_I@*`O;WESP_BDj>hi;W@9| zMS|H|RRI-&pI!bl57vf25WNT5ja=WOU|K}=Tx%#sffvw7_XA}yvXlyH@#6y?4;g|l zH+AH^bB3Me02{g~>u3mpS$S^8mS$*&c0dd9EhHIt*~S0x{4meR|9+{uV~2P3hxqn# zOYBsyreuYVLKAzgQvw6TZvx8qphV`(Zv@|KD=7<8?XSJV#jIs^Xoproc z&nKsgdXmn0e@xJLvpCs^_SBV1a`uK-UE?o&g{~_l-A#K}IX>NA33E)@&pragOMXSr zXkU+K->|r;Utevh=Z_ zP?!ar0B2_(OqPICMAI-2vPE058nC*)@85rSoVgtm9SXWAYkrXQ*=6e;W+=jh{idvV zN%oth^?j`x3lW0mxyc(UrYpL$WV6T{ertGTLf;x09QmQ*=#$py2p}+x;20>KDf}5S z)|1PJAybY3ak8^tgfG1XtpU-Fe3^b0Qe;u+jTFjOgA+lg2%+ZE?~)S%6^v)NwO3HZ z!`Xt>y`tCx^5$l~;5Zq4hU!?u_nW{NBc&IK5U(8x*-TCG4aze;gbEI30padc6ROtF zk)7A)E7=f1Q}{Tl+}<}i*t%HaNpoInn7Pd2U$%<~6Ac%#E#Gj-LW@D?wc$tQvSw*V zgsg3sznFwq7-zYBHq|Nbzh!A=!?NKPtr-l#Gn{7L-7{2gG-NaP8%LF{b)+*N1qa<( zuZ5Pco{;iY5YTV8C}gpmwGm}}4$`V)tzLsxSP@{m4y)art_7u0HDp-x=v7Q-ewDPD zk97_}ZCDh^7eX968Z==L2-}PISl(Y7O_=*Ek(eut*oVLvskda#Vhf2JBHi4xWRpxg z5@pV2levCr(rhKk$J|I3G$CfM0G-Hs9+iyYF;0sw zUed+I@db3fEx%~==H7shEv2Hu2o8#G;JDta<*e1A$r;Fy|S z2V{grzD`hVUWFoQTy6eA@XvZ8@1xaKFe49*pqTMH`hn2p$=H)!vgGAinz28ZzCBGb zSFJV6w8cVBj=K>g6@A_4v&Q{!PXCp=g+?UeaR`{%QUC=Dx<8}VnX2t$#f{8%eq~L| zA7X)w-u+hgN5wf_3$ZuJ?HOQcucEu0d%OH6$0H305;6SjxLA*jpwM^x(7$zQ6z(eY zd>o}a)zZ3knfFsdR1@fL(5??zQOu*$@Kw4jN8i7U^R5UDZW(kdXVadi#-LtE8iNQa z+}pe=pU=uV?n*%!5Uej%*Q*gKRxjlKgNOl4Z(<`9D%&Vlsj-ksG~!Q!7g;$f6Dx$3xG;}x!T7_Nd%&+M1{h%RhslAs#P}bYgs}81CKBZX(8Ih zd}n;|lq*>zLc$bE_SSB8nV+fWaFE~-HY4~#H8c2bTZNRN!UG~{bLDCUln^39!X%KX z(m-ky*G&sxnO=;GaXqRr;C;S);e>Qip~8h0zYZ9UTlsw1^Q-iuQPB0mVW3y_JPT3q zdiy?!pVXIfQrIRmYJP zauu6$#e(JhNirdr$PeX0Wh=oNuBfGQ#R@gcg_FIyxv0h0x|G2n!ynH!3hnQy=gKzC zIgW0smBkEY%7ruY2!wA~3E03D$32gFOg#w&{4{t{)p4oq-b4L1<(pK|gvn$si8o7% zD6z7LmfB(cIWh2AZy;aVtyDN;t>90IB{rbcobA=*U&h=5y|l&jmICp&O#ai~nhMOR z)pGq*J1@YR^^GZWl;&|E#!eZg-fYoBb|Y`JM?qjLaj~JXI1FW%wlMW8iUoWwTUZQ{ z_ZOG>*SBsn3IuCi>@tEqRr?&7EaWc~=KDXt@#xKn<)ixe&eKqe{2P71Ji#dfyctjM z%hiN{m$@}U$vS?YTZ;-{Yr$c~1S)s_Rs6ME-A$DE^?%pb=0Ejc|4;am|0*T>PcxkV z1M?(aibf157_j)tp>rr}I!mNej)d=15gAX~QR9K{QYL4*Z~yuwK%;`K8OI+@<4(H7 zc#GY9O7myd8UDOAUCk=8;Z>wdBomSD<{`c5b1V+6W(Rn}q-d4=;cT6O1nSpVfZkfS zn#$ADAx66&iD1l1j?fxm?(u3DIjlHc<($4gy|cQ4#qS(tZ39=j_iWAhrt0*c6HQM} zCrG%l?MXX!r2CF&|I9y!8WgUT)?_t>E5%0$){AnKZm0q~QZABwT+zi_L0#Kf&V{4T z|C7*www8>Zim>Uy&=gq0?wO_EkI`ei%y~|#|8dDn4K5T+$g?@sS9lm5;k7@*#2USb zdV_Iuzr5z@w+pvU!iOuDINfG2Ydh4_eS3$EJB-is3&SpFA^_kQuYo6TYl^)Z7+%+_ zoIGy~qR@Mf3eGYp?=Y9xl){B3aFChc!OYvp@}BL=|IkRtBNOlu+rU6y($vW}xQ3KC zDEH6SI8|yMyPR_R_b~+)1$H@8Jw9|tUk2NGew=!=m#9$V3f8co{MLdKG*xE!cs=z% zkXj(}#Y=_D-GnGV=5$1V{ISvB@t&pBYV>uPs5fG-T59!Fl>l{@c-cM*%riuVXA z&-+ZEgqp|mqBiFFYLDajy88<+Z}|@F;)-~_P5xnDf?hC{4cldcPpF6gYmkS)h1}^O zJ-LzxZp;{}R2>PHV{%nwtzH&>fL2H<61cdtjJ!zJojrL()<0E?yQ<2CY#H^q)*mti zjD;!{f;9t%%qEENReS?HYS1@Xn8m}wn#(gbPE}%9FQu899Rmu)*p*Rn zumoN&VH{LQtOZ`)*qkBFcFQS-hOoo?o5?yBJzrwza!M`^9Rs$d{q-MMec0(t3|5B! z(E4XOU<#M{+rb>r8=co$%i4Ln0QL`)%$bzCO(5s|GlyRM^qa%OU5T9d2Vcz<%UaTN z_)ar#7gXI-Q+HT3t=3H-upX@j;pUF%(zUv4AHzLl^ z_66PO%e`Ii2PqK(t}rMx3UcUGSEJU=E4sfY~=pwxYHoiba!DN>+AAw+)K+ zsJNLjK@Gn2YAg?hj{#r0ZQPXg^%FmLJ?rTG(_Iz4-hV?|^+mjdBIEPsB~%y=P6!N? z!@o~>G1$S&!WUCXfGi2S3ZSAV2kv4s&rfuE=m#{AdH|Df6xuCm1xtVuqF(cZj0mrf zyco%8FHN>*lMA0cvgmo-IvjfrJuz+O zb;3*%UrK|3ZCHxBxslc$$Yr06NFdGZ#o5{ePhZ%6s&z&=J#PF2#5@L0h6@FqP@z3t zJl=X9$5~*1$DOz^OZM9DnGh8xH(Cgqx-41e56bL2-yM`vm>c<;uIpqyI#(eK+qS5jtfXu%6PZ4#6 zZPH~sE}$7)|77cFAb85~)5&b`=euZc?S30XO?!Xf^2p!HDk*0S-*}~R*kCGQuMQ5u zF!)Hno@-0?oUD=6tj}4XIkFwL$(}mwNHG-Is`q0z21<3W2XJIEbW3I=}D%(>6rp=xiYj0&iJ$ z`u=IH6L`*keX@7Z_OBRWm3B=ykJqZvdpb^s_r+G2e#{eL3)P?JXcr8b36)>7&1BRI zS-B#r00`tOu7;V8o%*bi>T$b~%tJ{iH;qQm((3kmL_YZUX`xjp74bBynHiY$ed*9o7%m7NWVH_d1Pw8HX2h;X#yBYU||Z0drRA) zG-(dboZdfyyYu!I`$ZC}WDv&bTK)TxfUcqmF`>gY{PR+ks}hz9uHfrYOOxSjS2vFt z$;bhg^Cfut%yc!^s7K4pVb8VH?0-b@1Ou+;BuiH34$#Zu4W+ci>2Q_i!6Z; zj#5N`e*fefa&jc}Z#Zd3P>PGfGLl*)BEt7_8jdslK@A0gkv+6cW8Y$NS6|zw!WeO} zWGi?;-<+Ji5^N%vpwbS1PP{ek%&6#UKY2vlQ-st%QbrWGdb0=7Z*|MEK^yYd@%OT? z=AiC&Xxwd9f%;`o!YQ_y6J3{YqG^2LLg{)6G4<9!njumdngOxv$&!o-agCk83)@1Y z{fUn+yBlPfJCE7$S|*zZyA6CWr8bMz#OicOP#PVJz)n}k8fEJm)o-1td8)ofmZ>o3 zKTyd%cN+d{i4jz-8$J1Ph#Xbx!7QSPP-NtZ@=~u@?L@Vcv z)AdV++9SV6$UeM)xQS2{uxR$%!N>foh41H-pok}dS&7nPkRBfqF+Qw=1yJj+PbW1T z8Es7n)6C}+Yg%{ya(hKwgFzZo z9SFoVXh|DMX(&3_t;Un{uDfb81Zz` z7=1tN^{$NptF|2ZYCG|(eCSt#Yv&%i_-_s2Cbb5>r$Xy~2F!?^EkoYTYKj$>ZWnR9 zwn{T?VzVU3AeMIx2%T?U%-P60OJOm;CLW6ya4i>0iQak0mS}Ba_j7deD4Hw;{4<^@ z_V){xbSyW_7~gqH3zxht-6kzB8r=)ssyK;*UF^FE9LC81U_njevK+Cya)Ni38p5?( zuA7i~@TW)(%2$ABn&@2TbM%#LbRW~MC5GrQeP1ud8=N|p3yI8{Un-eW;}v8h;Qpa9 z_d`KJ{PDKqwzie^I44{mNyRC;0@$Uc1pBX|f@G3*mg4^g^7~&b;Ygrbd~l~9C>}TZ zN3=yUlxcjq+2i~1d^n!Rhlc|y^Dk?A5@ls&rCP5mZJ|nI@80`|;J=i{yRs+(rGA{p`^Fuhq=x__{g-{_vzWn(>f$+a?5^8$fB^ODWph+vU;llrJCM`~l$?Wf>oZ_r zU;yRO|IF*!;qdmElAgX6(J3q}>?X7l1@PBP0?MdjiYOt#K)dxF&J*$Ph3xI=&CJYN z)-v+*6@au^puqax9M6rL04fBuMf*f?Xsb-wR38oK{}cm>&<9ibQ$V!rsajtH`1>{x z3tYc;4xBNNf7%1fYu%DHJ}wIstc!<0A(H)y>XZAcq25I(qJ-n|xcCDUpI4+O4@roL zIqigD0ins5X>>Z_8}@Y?YaRvpTWT75Vxpp<_|yNK?HswqTD=&+ULg=}iGa_E0wk9@ zv`&~ZCuL+vj-}B7Ii3nV*7Wq$)VlzpfWH7@baZqSLc)0IAK$_Mc2N|LY!aeG|4Kmt z9f(P%Jq?5-yK?LGtjPnR#ipjFJ*%a$u?U#-x{~VZn84LxwVH*96DNgkdCHdx=|%x6 z;($C`_bvk9)j;a9cFlTBJ{3@FMMXs=C|8p+F8PlZc1R0I3q{DEikL7brKOe4+M6V% zPK`b~vIGILGtHjR|Jm-zZ@?`9LO%auiZyAKi7hlEn^tT=fOKOZCJU(A5zdj*rPny0 z#`~5m1@{L5-U)~_Ew{p~Q$V)+*Kw2Y2hs=-pn#nUXNfKl5S+Yw2+K2*gu|^c{t>6s zq9Q;kp+f=(ufU!cbP~>45%rHZv`f2t?FpoKE2DWpRqCJtj{;B^`mc3~)31{#n;T88 zu-FyTs2MS2G-Sm!`vmTm{odc*ngxP+ftXQPSlHPdZ>Z?__=E2<^g?BGM(o)?1c$bY zluP(E{=zYLV@W4~W*YFXe|w)p0z!ZB5g%&nA1FvuqY0*UbDc9*ycWX*ZLo>8SNDM< z2)I=#A%9(jas&TO)ya2(ORnSnF7~(Z_~YbLzYfugZS&lm=F_l}HR5;Jf4RR(eeJ$N zH|HP8`n}Das9hX=o9Qw7kfDi{?JvZJ>C7l)|0o!Nqxg?r>c5mx{~p`_!N~EpFM9~TYY z1Dza*#)U}5tP#JM_!lV@?5f`Bz=!u^8*s+GX?(rm<-hKtJ3a8Th=_mtc!@&k%=!GY zIKIB#HH%}aVCp&Kc&!HS7?ufGt{zYo2pF!Us$;o;=6UqxPP>=^WLPLEU8!L;c`YRP z-tM9E#yK1w<4uxIQXAO)GuI(4hERaM^uxost-z-7BzomKe}AOQ>o@m0y|Kb%Av@y* z>L>4?Wvu6Zd<=az*n`~U#R6w`!5?rRId@!-y!`QeL51u12n4sJ7#>Jy20nRZBLU-i z&KTFZatOGN^oMOJqb;Bao$jm(XJcP}6soFda;5Xd+XhSc{rHl7ubU;xI;2@<%yJcQ zzTx4is`df6e$3sR#j_$KLGey43XW9Pqb-kw9MfKGeQBbpty;#yJE878(r%!M3=S(L z{yM8eNz(IZ`h8rsbm}r83WXS!cpJ>$V2UMyxdRBd?RZ?j?KoWDtt}bq40g77&YPl& zPAWO7D6y5pns(S{)SF zwLpynw^J7AN5!fp_~+R$mS1wgE;!s_KnXSrArc=_8+;ow0CC;{#Dd;+9YFv0V97&u z)^mQ_`ryR6Lln457~u?eO%MxnnbJA{Xex{;G)+^?25m8Ew3WIFS$(LZCbIbG%W z^Kwlxh7k-D%%7-s6&2I>j+dePh6*pbMs%bNZ|lm7T8saci!h%nLbxn1_FCiiS5KRx zx(2MHk9w3nnZ<)=l)OJQVV{MA$~ldNi2WC_^Dxd~;j&+N8HrBxBY^}Cz}J;$?mS10 zq1;7swZnQimSV-YtBh~Tb!l7*huse$7hDk)B2q5}?xjMNQ2O)UZ}t!)WL06O}AEYLR`|A9#|>hFeN|j zJn|~AR9IEGr!yas;x_D?)}Si*5`L(LK|xp${(eX^1)&|=E1#0azmLM4v-xn8ms~G? zl@Zqy!^$2Kc`V$qdZy$IXK7CVwix)@OxeRVhMfV0*6%&md>!}khLP}373)1I7KAwi z>t=n*vwu$7Lk3O_2j4!gevP+L-eGgzR#Ig0pG$|AMHDHGd9NCHJrDY7F(R)y5D0@3 z(JLTcAn_L~2WhKKt&YoRY30j+utm*0I?K16;^gX!%O4ERdp=xJ-6}3Uf#%nKVoo@e zF=f?luj)Sro~!hZ`KQ5MI>Z=&1D;q3~PD*P5_Pd2hndq-x2;3VTxD|Lp9{4)H*c71 zu+0UOL=a$x5SE1dY*BKS$Dw)es^yOPsT;jIV@&3jBaz9MHo4i$ zx5HPWtEoxNc0bp`;#NkA1)RP!3hVRU;{_na)CM@M89AXX81-n+zuR+7S6sqJP>nxX zI$k}tmS=MRCi3s2|V%;p;MZM$i9Sy!;fe8lbduEFpYyYpB$(GYq=-39^F~M2`QPFzENCPQ00UUA6><{iHeG`w;BOk5D%ruGMa0m{# zJ?e2^s}KSc6J@ySF3DV@2_tyYt8b8(_4|-6jdJ0Bh*u$r)xjCI*pu-DSPrBSFhy(EwI>@=r z6ts%4sckvQh5Ln{Q@0WhQUrIowKX6<@9#UA)9ZNysd15h?GOyQhAFgHgKITs)1->j zWg>yLf@7!opz$Sp%G2Qk^KF%JB$hkw$ zkwfT{<#tDYymfuw{_j5om;+{%KQ1dQ+Wln}WU~Yw&&9zwoqSFdZN{;|apx;3cL&3h z2^Wd>Cdafi4CDd1F4|v3s z+WXUG$|R9OX9L{c(2eJO>mmDzpd0MW)}9O#wECe4PkRGg(m=hZhaoPdSIym(gXl4S zjzC0kO_-2$^30EicUBi)fj1Vu_`vkRE9XJqk4@KO=A=6F{k1xV@+*I8e&ssdJ;j+Z zyBr3$uF0G85pU+|sc~ePT6brfYR-!o7>|^(-;bvOpqN?1YDEW+7$T@kJ_=zLk>uBC zslR~+DnjLZFdg2>j9Ad^uX*sBzmGV0evCrAnDYY4>5T$T?)M$vhZM9tW~SQ{m30Vk z?cn&(i1u>mu^1UYoSktkl;@11;VlH)+Hj^(Vf{9lr7Qb{i;;wC%C8}PC7OK+Jupvi zFNLo8`=_6*bO~L)bXoV3Hl2aHFWLzlm0khJb`C;92`~x*`}{Aj4mJn#)aP8M5yJFv zeIb*HmN|l#ptBOeLv}7<#^xu&zw!-jf#6&U&mk-FAyZ=o0>1IE1C7#10q`2G?=u>k z6W|2;%hQ!!b4RcV(IeF=Mh%)rGbY#;@>xHJ8U`if68`+E6;e#%ez4BVzGfut6yI^E zwGfRC@7sLFP1V6{bY2qpxFea|+ZcXm)bsf~r!ti|4mif-2m&n}gy9%cG^x8VL{ewJ z_z+m+Kt~fK8Bto)00AMrmJ$_G{mmQxA(hV532=<M=+9ju|V_w)W_plbX)!)9!PmDQo|Yc2sKi1r-Z z#gHw(9qX|p7h2QSU$*P@gYL0zvS~Wn0cdw}?KyG_33IOKz)FLOvij1**-o*u$aji# z*S^#_RC=;9C#(g+ODFw;yloi_Xm7N39YmJXqF@~Q?TZ4ADdfDbmb!%9^`LC5v`}VW zhGc_I90q#MDQU~V@ofsdXy)lcn>k2cE)^fS=2InyQ7Fi~_!Ce7jV{79!>E{2z>Ww> zNlBrh88cR@loHOvLY829sIldiMa@O{Wv6W)G^xr5bBJ^z1uUD%*eu0iBSuh$WuDggTHDk9p9jUIS1>nKPWe92%hu@Umvvyh| z+xX+E(NY7=mxpAE&{^fOu=yqi^0BmF<0pnv_L)ol8#{b!9(v{e-?4 zc}`iTnK|j4W4P899yRgWSFYX+(aQib6K;5+tmR1xC`Gd-ixl3(LOoj>JMT z>!jWd#a;crTTVeaXE9$_pdH}`fN$Jmn5Owlg>)T5P?|AlVg>GfjqCzPhiBCyd6R~T ziI~MPZ1Y%8FSvx9(SWE6Q6v`}GOgDvxZ$h6zOB77%6Lj9$>s^0>9tYg{=Z22J2 zE#gf1uW>p=L_yYy%pvfMdJVp{y@0$}&sT*arj#^uD*J8Ak)}P z@b?^o244SZLFqMx`kyINy>oFS|4v`Z9+muJ4@dha-~E(D-+}LqE4KA%L_4!ol%UA= z2(I#P0#KIs<@IxxI#c0q7Mgm!=&w=EY9WUfzw(hrIYi2i6AeK;zNqp;@53W)B825( z`GQX4`?_0?3KZO+A>Eq$+oiGi(_q&HE{}l(Tsi5weFl9$>^Dm%h~_k+3KA5&NFf!C zc;h^wp`x!HshRnbo+ED?**cgPH4+VcfL24?VWNKLL382^qTU`<*OAuG#Ax}JGl>%w z+WbsQN2G7TH*BV!Wjl~ON=Gs?Gz>6i&uIs}IA=@M{j|CuQ3w!kTggfL@aG9&Gnp4N znbB6Hq3cQA)W+3Q^W^EiP>Ive7R}&y&3BOp1ErAnAeYMiTlz-zX`2Pcd`TsHG|B3(rt-pVk;r~9*f8X?9!G!<((!ZDN zKN<7?TG9WfoBr=%i=ik{QegJoa*WRaqn>lj``4TT?h{JV!B?tw6Efi-8nyzZ5?X(( zG2)}^ynjVjIG-`J^i0(`k-T085WAe9olCakztn+Qg2FoWHMPJ4BPJ9CIPw_(^A`Wt zc>im?{xiw{S})*DPESDB)x8U`4dC%V0y7-U=n<^wlBuglKv4Zq@w?Mt7ov1mICVs@&E zcd)BgOWz)Sh%_DA<{a?*DcQ1wS>Nv9{K-=}y?X=|h1q21my>O#K* z%Em$AhK+13tisd5_ZVU_7;$P(Lbd8P|K3ozi7ka{28`@SXopMe#nn_o=I(rq@k8j9 zVASJ8opw&f@lGsSV}KTV7+{ z%Tw`Jt>x(}r?Rlo!MRT^WbOlhL3fa~=_uOC{Mr9y&W{WEdE0JmHZT|rBzGUcX~WLu z;=kU+*wZ>=Y;PamP|9aB@8FrGYM%dME|(9C2EbXdlh+@;f&czcc1R_YaQ5H$^vxF; z(dEDe-uBH0+%fIn^t|=oJbUM8!rGfjb1gqDe4W|veaMO;1J^z9IuBkl6qD}Yoy*?h zUwPu8dsyH+jq#US02>>>dYx%c|CjC6uDv17U?9Hx049vMoEz@DiqlgfTK-VOvRRLF z_d82y=-_+eM%~Y{FCV3M_<<#A&ELUmkKWFwyN2@dnwj)!H!XN5-|++Qy!9dIRwK|M`X>T#kw3gXh z?!>0QfC+t`Cq>usqB-kk@zU0((Hk!0jEp<=rNSR*q`qM#-|f1aZH>E+qr!}Z zxap8Qw%r2-YF>ftZO~TdwDCgdF&*@+wq>6KnwCS+45(kZ_x-55A#n)gUjcOrba@9t zV-HfGt03!dV2_6`k3;kj(C7dsxQihFUZ_|g&595b;{G`PS}XOnel(^q;$x#RXjS;! zHj37K%gim?DfYw6JI*J@xNlF5E;O3N=x{XZ{kp}p7LsB@Py>fGau)E@kiU*+AHIt> zzT8NY+DvRz7vkep_`OaV3b*jl_6;oBvYY8oJ<0W>GSKb+-2oq!IqUdt**eNg^t|`M z62|tsoM<(uv_|3*)9BXCytmJ;shYwHE1)GhF_8$<-afUsq&N&(WuN79*(=%f?T1X8 z{xYj_^z>~1$KYuyV&3zQ^4RO2lilDYvgdHZEXG6XiS#&f`S|&#`Erw!OP{`hp($p- z1x-y@D_x*95SN%hs6mb2>%yA5kvTiE$#*z;`PsYZ7rk%apxPKhr`Xsw!((((G&f65`_K-1J_~>Ogu8Elg73HIKtvkpCId!Y>t#?d$7XQ(b(gU6 z!MAvFW>0#)cr(dH^$A~pLQEGizWd+leC){Dkhe)n>k!%QJ2w zV4KIX{QpqxZP^OFRd_YsIJ^53^fDH+D(f0{HDnQHIF0161nj=zXvO*+Z&Rf z08>3=-v+Lh-h00doK=uA8=|{G)eNXw1WG)FPXzZ)X;6fa5cj{)Y4}t=p7l5b6OGs_ zH!jETU8Q?K#P~@P59WaT5ul`>h^DL9N!J18NWB`+crj7QOj6 z&wlbN>bQPfb=Q5|HDx@hF$P?X1*}{AKGPrk4{O%WK8+#Qy-Gky<-WJR=C(6B@BJBS z4La>_LBvIQ_Er|W^>3bg_Y-ofyaeddelYemvGTJQnf~~AGM;_p;9ua6% z%E1bP?caXDt8>3c7cqhJ#ty~2-*|0E+8`c(^E0j-9D%#3n0Yh)k7=(jX2p-+uyo@^ z^d8rh*6}FjlnZ!c)=WlqH6PgSskP|$bdUORG?uY_$+!ILxyMZ=wsUsaoEAJ zxz$%H;B`U32KEimY;QH~2#8Dv`%)-<5ggWFdTjU_j=S;{;`ksJHvAFIky?Y^V8moH z5tBBY;X~ulXo5X;R`-GDpQzM23@y{|^D-LI>oh2b`TdT(pLuisuXz0d22c7s({GtV zuasy^CJPar`Y`V52bp%w1hgtBUpUF4p3tUy(c=4%e{B!2V@T!pxeg!uU_GX+aF}Xno>Oej(@4UzKBm>ewNQR=VD6f z!|hMM%>z@06Kc|NU{+Nyckac={$0`37qDR77woQfo#3@5fY)6>qsKs8)EMH7!LtTU zmJA|PKAPOsxV(qiGam5Z4+Lm-7Gd|hk3z)`ZgV=dsimtbSU3Cu&q252#z z^cp^xw1|)c!$y5MyLW9TGc$9~E4R1`r}y`KyN_)@e!3W3B7wWFeB-J3oX13D+Sp*_gPn2IR!NBpP{^N zY6^>~Gxg-QXJ2Fbl|$N(FMby}Yv=RcyT4%5#xVA;w=uDshmzbK6qQud>~5p`NuNNU zeo2^3F1D>+&M(`Fa0gE8dedlM$j?RJqq0ol!oD9cD&-%HNuPqo{tIhLKfrp>%%Q<- z>_(iqH=Qiw7?$=Z>B=vxEc%#wkK;&dHI2???5-&!JmzjDcDa^hV2{gE2x(MQW zLG6do(p!1Z=0ckQrruyW6%w9>Fg;Yh(M~Vi;o1n*8^AOk(jJA7-k>oegy?85Cyu#_ zpFiZ%u^;X|0M?r5bM9?C^3R*-5^Xr3XKnYF&oKJSXZCq-&=rp`=jF%f6@A!EA*F>i zc-(%>Dd}{Nk4Do*bhktgWY8&UmM(^@tUQ_l)1F!Eg?-zaC9?wZGM%d~mva-V^QNIrxsU#xXVN?28a7va&bIQe$ZyQUdt8@pCrG#o$b+f{ z;60R3YlOH*gD(JVP`?1|6`<&V<|AAt8)IId z)EP+~IE+*PTX}~+)mgob6~6>ejQ6=KDJ{^VF&I$$eQf#lD|YAZpx*dCPhLK3p9@gJ z5>rUj8OW!O{Nl3UJ@E7=?gb8J<2j7$F`XXT8dmOplzLL>6@NWxiMKJ_SD2j?m(;P(3P`P<&h1_B zC**+i?rJWkC_5P9V!EaT&ld~$smxlOKKu*{^Q-~*az zaDL0?s+;NGr4NbvSaNJ7hdgZQ^;@a1Wz*zw5N%4s!jTUEw(3H{=`JR({NL>CbOEPW z`eD?D1#eV)Tv3DhY={~O?(d;y)e+3A?#45+?+l=%%WiS@gGHQ!!S#G z8nFAC4{P*JQ9_Bg^e0?z!ChZ<6at(IwaS1JaN`Yl0W}(R2pUDf=W_>7ojBeop%8iw zm_niOgSKz%dpLnm+XE~~;8}Hat|C>81h0B+nU`V#m9FJIIVVmC332~p9*={9%oVI! z7rdHFmytxAa%>yim|+w6OX|D4z9ENqU!Be1j0nc}?TS(D$8EDxvh91Ge|{e2ZWYmk z|H8Oak^lv7!zMm{ZzT?NbV~0|T6{RIvmed1Y~7hd>6VquoxhUP?>U#KW3@%8jU=U~ z6W2+@Hft#xHtyu?e3Ii@Y?t?e5KA#(F z>1LKK-;CGihqQPS%m%=5OxKszl1A^op?qFk%k~YMDXAMu&rTLRjWuNL*hv*&&_|P& zyx*0)Za2*>-EZj$ozUyFCvZ(Erl4*;?#K%{EA0t9+SjOT){qc;HD|`8P+R{NCC;*g zgg7nXmcgX@I-w{kqD{1Qtl+VtHZ^o*8s;<`cHji7fx*}LGWKH>vT;%0!+ z3f46cP$1$ui0K!+kDaYR8W15Q#QpIabJy|qE6=hhOpC)&!=|r4WbqCgkjTit3?o{l z9MK8E*~>p=hJVw3y)O!(vAwup>LilF2XN&bm$3M@=gD3D9)JH^F_)b)l1RNDTS+b} zmweB<{7MY*1G)9?TN#`bfucCcTktuWjT@eNn+GQk#?-pitJ%oyS50O1;$19T zyol^8MiH58;n+k{9bJZuW8BEk*!sl|-gxXzYB!!vyxvRJnr~UKqY!meU(P=3Y&wOh z+H8Qj4Lpr-)?*9QP+Qk_uYs$!n1%1oq-?1Xx7$YEFAG@k(@wnp2nL;f23?~=P>ykk z(;1T)J>^2?{5+G*pS{dosv^!vkHJyCleu%hrWzPD^fE?v+jl6{UtdeH*8{{duy4HlRG|N6;;-s z__^Q}eyPnr!h5+XXjNuR#$NPKp3F#%4i(MV?AzF0G>xB2=F)T=cEn*k4?-+ZzXaTe z8@db-dw=i+!Gjt5W+=KFD!&5Xk^W#4=whJL4ba&Hc)?Qyb+e&(26%i3Atc28;Tp16 z^ZIkETmGd&tJM?Ixfc^IxrcvTJPfr;!G8q%zpSNmn7(xXcU5%mdl>^RoJ41nodWmRnlF=tI7^ZxS`^I95el}d*>GLbXS z`5TYlbp?rG$NrujDy@OR6L03how>~TU@1S$nZ+Wnm*7(j%yd6>5|7<~F^R38?v9_@ z>PoyG51P0^^y`s~L3?7?8@FdGYjbZTzaouLUGP>sUY~wp*SyjE1!VpTuASgLi~(iGXRv)ED6Mw_Zg7`?WB0zdm8_Wo zO<~a7#`$ZWEGWDV)NXL?e`-Q4luZNce?hGS{NS$x$1d>Lq#+SPLfjw8MCzbZ8RL!E z_enQIk)APtb1y%S)BB|nV$!1kiV{N4;b$_wb2VvwQZQ&b4nkUE9D_!W!Q*hX+fxve z){h8n@c#{=$y{*bQ}j9OLY9B_2|umNryekyqZu}4GH0K28a)%DG3$amX=_;tP3CCM z7<(Q)&z?lz)bKXWchk{l>?K_GQv>@>$Q zY{XfFbT-gCC9;i;Q%n4i^Lg>zB*uNckon7YVYl0fh)ChA3oqxwal?q|;9`*Il+!u) z!k(=EJc|{-ZpJ;i2fCK|#-#V<%<<#dw{OBsbaW5KOrFTZQT>SwHMb1E8A$3gn9<`^ z===61%5<33NUbNa#}LMk*VCy>588NO7lx>GranB2K@%48<|pgOD=0yuF)-lNbD4C> zL3!1Bp73?ZkKbQNWm6%QhZp~_H*X;45Hd<2;HI|e z$Mzp%=PRbLzPR1*X?A8(?94oIiqyS5_&Qv^e6Swk4Sk(kz}bGl#`aVDpY;|&W05o; zLL4F`5C{aM7{s4~&%x$-FL1-%&(n}`BVT>}9KFmZ$#1i6=Amn@=Rd2Oxc!x{xcBm& zSdMU(m%Nw77h4YfaY}zZ{TgCJb7VU^MxqtCOA8uB%Hnqk8vz4WN;!y=irgno{CTKN z^f~7et{p!N`|f3Ywk!*e|0MWbeka?1`i>vA6w&oB*K+j*M;_pW5JE_Z6XLj{v1IVq zM;>A7pfDDH@F|&9&J+9>1#s0D^5sVhvBzA>laJg$s!7(f5JE_Z6XGP?Z>My16_r4~YnKMA?8LHzkBDx$iM;Gz2mpwXZBC&h)u z_v7ZrdlDKIidxpL5JE_Z6XK+%(hwFFeljFdv^paZQASy(LJ09^Ys#bugb+dqAtb~J zA%qY@2nlgQ2qA+qIxML(-0Y&!M zzn`&(lz}+u$YZyJ5JCtcPCzo8DufV12q7U(2qA`_ installation +.. toctree:: + :maxdepth: 1 + :caption: Precision and Performance + + precision_performance + .. toctree:: :maxdepth: 1 :caption: Contributing diff --git a/docs/source/precision_performance.md b/docs/source/precision_performance.md new file mode 100644 index 0000000000..6e6c51d8c1 --- /dev/null +++ b/docs/source/precision_performance.md @@ -0,0 +1,39 @@ +# Precision and Performance + +Modern GPU architectures usually can use reduced precision tensor data or computational operations to save memory and increase throughput. However, in some cases, the reduced precision will cause numerical stability issues, and further cause reproducibility issues. Therefore, please ensure that you are using appropriate precision. + + + +## TensorFloat-32 (TF32) + +### Introduction + +NVIDIA introduced a new math mode TensorFloat-32 (TF32) for NVIDIA Ampere GPUs and above, see [Accelerating AI Training with NVIDIA TF32 Tensor Cores](https://developer.nvidia.com/blog/accelerating-ai-training-with-tf32-tensor-cores/), [TRAINING NEURAL NETWORKS +WITH TENSOR CORES](https://nvlabs.github.io/eccv2020-mixed-precision-tutorial/files/dusan_stosic-training-neural-networks-with-tensor-cores.pdf), [CUDA 11](https://developer.nvidia.com/blog/cuda-11-features-revealed/) and [Ampere architecture](https://developer.nvidia.com/blog/nvidia-ampere-architecture-in-depth/). + +TF32 adopts 8 exponent bits, 10 bits of mantissa, and one sign bit. + +![Precision options used for AI training.](../images/precision_options.png) + +### Potential Impact + +Although NVIDIA has shown that TF32 mode can reach the same accuracy and convergence as float32 for most AI workloads, some users still find some significant effect on their applications, see [PyTorch and TensorFloat32](https://dev-discuss.pytorch.org/t/pytorch-and-tensorfloat32/504). Users who need high-precision matrix operation, such as traditional computer graphics operation and kernel method, may be affected by TF32 precision. + +Note that all operations that use `cuda.matmul` may be affected +by TF32 mode so the impact is very wide. + +### Settings + +[PyTorch TF32](https://pytorch.org/docs/stable/notes/cuda.html#tensorfloat-32-tf32-on-ampere-devices) default value: +```python +torch.backends.cuda.matmul.allow_tf32 = False # in PyTorch 1.12 and later. +torch.backends.cudnn.allow_tf32 = True +``` +Please note that there are environment variables that can override the flags above. For example, the environment variables mentioned in [Accelerating AI Training with NVIDIA TF32 Tensor Cores](https://developer.nvidia.com/blog/accelerating-ai-training-with-tf32-tensor-cores/) and `TORCH_ALLOW_TF32_CUBLAS_OVERRIDE` used by PyTorch. Thus, in some cases, the flags may be accidentally changed or overridden. + +We recommend that users print out these two flags for confirmation when unsure. + +If you are using an [NGC PyTorch container](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch), the container includes a layer `ENV TORCH_ALLOW_TF32_CUBLAS_OVERRIDE=1`. +The default value `torch.backends.cuda.matmul.allow_tf32` will be overridden to `True`. + +If you can confirm through experiments that your model has no accuracy or convergence issues in TF32 mode and you have NVIDIA Ampere GPUs or above, you can set the two flags above to `True` to speed up your model.