From cfccb9bfaa1cae42551b59bd3fcf3713a0392126 Mon Sep 17 00:00:00 2001 From: adrhill Date: Mon, 19 Aug 2024 16:33:00 +0200 Subject: [PATCH 01/13] Add dev docs --- docs/make.jl | 9 +++++- docs/src/dev/api.md | 55 ++++++++++++++++++++++++++++++++++++ docs/src/dev/how_it_works.md | 31 ++++++++++++++++++++ docs/src/{ => user}/api.md | 22 +-------------- 4 files changed, 95 insertions(+), 22 deletions(-) create mode 100644 docs/src/dev/api.md create mode 100644 docs/src/dev/how_it_works.md rename docs/src/{ => user}/api.md (51%) diff --git a/docs/make.jl b/docs/make.jl index 69154d14..373e53d3 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -20,7 +20,14 @@ makedocs(; edit_link = "main", assets = String[], ), - pages=["Home" => "index.md", "API Reference" => "api.md"], + pages=[ + "Getting Started" => "index.md", + "User Documentation" => ["API Reference" => "user/api.md"], + "Developer Documentation" => [ + "How SCT works" => "dev/how_it_works.md", + "Internals Reference" => "dev/api.md", + ], + ], warnonly=[:missing_docs], ) diff --git a/docs/src/dev/api.md b/docs/src/dev/api.md new file mode 100644 index 00000000..954245d8 --- /dev/null +++ b/docs/src/dev/api.md @@ -0,0 +1,55 @@ +# [Internals Reference](@id internal-api) + +!!! warning "Internals may change" + This part of the developer documentation exclusively refers to internals that may change without warning in a future release of SparseConnectivityTracer. + Anything written on this page should be treated as if it was undocumented. + Only functionality that is exported or part of the [user documentation](@ref api) adheres to semantic versioning. + + +```@index +``` + +## Tracer Types + +```@docs +SparseConnectivityTracer.AbstractTracer +SparseConnectivityTracer.GradientTracer +SparseConnectivityTracer.HessianTracer +SparseConnectivityTracer.Dual +``` + +## Patterns + +```@docs +SparseConnectivityTracer.AbstractPattern +``` + +### Gradient Patterns + +```@docs +SparseConnectivityTracer.AbstractGradientPattern +SparseConnectivityTracer.IndexSetGradientPattern +``` + +### Hessian Patterns + +```@docs +SparseConnectivityTracer.AbstractHessianPattern +SparseConnectivityTracer.IndexSetHessianPattern +SparseConnectivityTracer.DictHessianPattern +``` + +### Traits + +```@docs +SparseConnectivityTracer.shared +``` + +### Utilities + +```@docs +SparseConnectivityTracer.gradient +SparseConnectivityTracer.hessian +SparseConnectivityTracer.myempty +SparseConnectivityTracer.create_patterns +``` \ No newline at end of file diff --git a/docs/src/dev/how_it_works.md b/docs/src/dev/how_it_works.md new file mode 100644 index 00000000..2bc681ac --- /dev/null +++ b/docs/src/dev/how_it_works.md @@ -0,0 +1,31 @@ +# How SparseConnectivityTracer works + +!!! warning "Internals may change" + The developer documentation might refer to internals that may change without warning in a future release of SparseConnectivityTracer. + Only functionality that is exported or part of the [user documentation](@ref api) adheres to semantic versioning. + + +SparseConnectivityTracer works by pushing `Real` number types called tracers through generic functions. +Currently, two tracer types are provided: + +* [`GradientTracer`](@ref SparseConnectivityTracer.GradientTracer): used for Jacobian sparsity patterns +* [`HessianTracer`](@ref SparseConnectivityTracer.HessianTracer): used for Hessian sparsity patterns + +When used alone, these tracers compute [**global** sparsity patterns](@ref TracerSparsityDetector). +Alternatively, these can be used inside of a dual number type [`Dual`](@ref SparseConnectivityTracer.Dual), +which keeps track of the primal computation and allows tracing through comparisons and control flow. +This is how [**local** spasity patterns](@ref TracerLocalSparsityDetector) are computed. + +!!! tip "Tip: SparseConnectivityTracer as binary ForwardDiff" + SparseConnectivityTracer's `Dual{T, GradientTracer}` can be thought of as a binary version of [ForwardDiff](https://github.com/JuliaDiff/ForwardDiff.jl)'s own `Dual` number type. + This is a good mental model for SparseConnectivityTracer if you are already familiar with ForwardDiff and its limitations. + + +## Index Sets + +Let's take a look at a scalar function $f: \mathbb{R}^n \rightarrow \mathbb{R}$. +The gradient is defined as the vector $\frac{\partial f}{\partial x_i}$ +and the Hessian as the matrix $\frac{\partial^2 f}{\partial x_i \partial x_j}$ for a given input $x\in\mathbb{R}^n$. + + +## Operator overloading: Toy example \ No newline at end of file diff --git a/docs/src/api.md b/docs/src/user/api.md similarity index 51% rename from docs/src/api.md rename to docs/src/user/api.md index 8d8564c2..37f00569 100644 --- a/docs/src/api.md +++ b/docs/src/user/api.md @@ -4,7 +4,7 @@ CurrentModule = Main CollapsedDocStrings = true ``` -# API Reference +# [API Reference](@id api) ```@index ``` @@ -22,23 +22,3 @@ To compute **local** sparsity patterns of `f(x)` at a specific input `x`, use ```@docs TracerLocalSparsityDetector ``` - -## Internals - -!!! warning - Internals may change without warning in a future release of SparseConnectivityTracer. - -SparseConnectivityTracer works by pushing `Real` number types called tracers through generic functions. -Currently, two tracer types are provided: - -```@docs -SparseConnectivityTracer.GradientTracer -SparseConnectivityTracer.HessianTracer -``` - -These can be used alone or inside of the dual number type `Dual`, -which keeps track of the primal computation and allows tracing through comparisons and control flow: - -```@docs -SparseConnectivityTracer.Dual -``` From 5c45d2bfabb42d00dcdc9fda99826712f59778de Mon Sep 17 00:00:00 2001 From: adrhill Date: Mon, 19 Aug 2024 16:33:08 +0200 Subject: [PATCH 02/13] Update docstrings --- src/patterns.jl | 15 +++++++-------- src/tracers.jl | 14 ++++++++++++++ 2 files changed, 21 insertions(+), 8 deletions(-) diff --git a/src/patterns.jl b/src/patterns.jl index a07756aa..c8d4a707 100644 --- a/src/patterns.jl +++ b/src/patterns.jl @@ -36,10 +36,9 @@ Base.Bool(::Shared) = true Base.Bool(::NotShared) = false """ - myempty(T) - myempty(tracer) - myempty(pattern) - + myempty(T) + myempty(tracer::AbstractTracer) + myempty(pattern::AbstractPattern) Constructor for an empty tracer or pattern of type `T` representing a new number (usually an empty pattern). """ @@ -53,14 +52,14 @@ Convenience constructor for patterns of type `P` for multiple inputs `xs` and th create_patterns """ - gradient(pattern) + gradient(pattern::AbstractTracer) Return a representation of non-zero values ``∇f(x)_{i} ≠ 0`` in the gradient. """ gradient """ - hessian(pattern) + hessian(pattern::HessianTracer) Return a representation of non-zero values ``∇²f(x)_{ij} ≠ 0`` in the Hessian. """ @@ -160,7 +159,7 @@ For use with [`GradientTracer`](@ref). * [`myempty`](@ref) * [`create_patterns`](@ref) * [`gradient`](@ref) -* [`isshared`](@ref) in case the pattern is shared (mutates). Defaults to false. +* [`shared`](@ref) """ abstract type AbstractGradientPattern <: AbstractPattern end @@ -208,7 +207,7 @@ For use with [`HessianTracer`](@ref). * [`create_patterns`](@ref) * [`gradient`](@ref) * [`hessian`](@ref) -* [`shared`](@ref) in case the pattern is shared (mutates). Defaults to `NotShared()`. +* [`shared`](@ref) """ abstract type AbstractHessianPattern <: AbstractPattern end diff --git a/src/tracers.jl b/src/tracers.jl index 750da3d5..4bf041b0 100644 --- a/src/tracers.jl +++ b/src/tracers.jl @@ -1,3 +1,17 @@ +""" + AbstractTracer + +Abstract supertype of tracers. + +## Type hierarchy +``` +AbstractTracer +├── GradientTracer +└── HessianTracer +``` + +Note that [`Dual`](@ref) is not an `AbstractTracer`. +""" abstract type AbstractTracer{P<:AbstractPattern} <: Real end #================# From 77b92b9d5ef5847786f15c1100fc26d7621a642a Mon Sep 17 00:00:00 2001 From: adrhill Date: Mon, 19 Aug 2024 17:30:02 +0200 Subject: [PATCH 03/13] Add DI favicon --- docs/make.jl | 2 +- docs/src/assets/favicon.ico | Bin 0 -> 121769 bytes 2 files changed, 1 insertion(+), 1 deletion(-) create mode 100644 docs/src/assets/favicon.ico diff --git a/docs/make.jl b/docs/make.jl index 373e53d3..3677d206 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -18,7 +18,7 @@ makedocs(; format=Documenter.HTML(; canonical = "https://adrhill.github.io/SparseConnectivityTracer.jl", edit_link = "main", - assets = String[], + assets = ["assets/favicon.ico"], ), pages=[ "Getting Started" => "index.md", diff --git a/docs/src/assets/favicon.ico b/docs/src/assets/favicon.ico new file mode 100644 index 0000000000000000000000000000000000000000..92c38a40307bef6025a867a81f0f0f8f74ac9f60 GIT binary patch literal 121769 zcmZsCby!?Iu=g&DySufxYjH2d-6^)X7I$~oBE_v}vBj;xBE{XMg~bZR-R1J$`~CTy z=bR_Wk(^{E^P5RB0{{R42!Q_%Ab=X+mI(mpyyb&~|F6u533XE^ z=_PP+<$`G?kSXB!P`dcdL4(V#rx|SeU5abB8CiR`L%+FsyiWe4H_g`OJ@0>5&Em2C zJDJNf<(BW651PvypXui^(Uo|Dhe$Gv1ksyI#Z!uS5Dd^(sis;G& zcLK@)FDTPY@2OZ201vdtT|U1IB@`8Gm6@DGGK$v2szx|O{BTu>Pt~IR$e{t|J@w&& z@1SZk7!-{>;r>l=O!54P7NO!Cc)T^#?F(+mu_qmhwngNOfE*#bJEtN3mo|ol$Rl|e zyeH>LV54kFT}vZ4TYhbmjXbR+6}?UEj5m9m{3p?Yr?$;RQkKNuHIT}3Y2^D7MfOoy z&Y-$kHc$zNwbPYPPuX%RO_Y!sCK$^9ei4pB+Wn!-tP58-;Y!T2ZfT0Fh^-LtJMm7a z;dJb`jw#Kb?~hL*PY%f-@MZH{e%KIMSwgTTR(|hi>z_C-bdW-`38d`ks~L*Y>DEDk z#C*Fn4iPmZ2&LYaGNRya%ViH^sAm~QM3Vh~ga(Y&W>b7k(Px;|RQ9Dib06r_su%lQ z;SztsVdlXi3qNymD-45J>O1_IL_L}kWLtiafgEq;1 zRgWuV_C08f%w`R+IV8rFD2~4mRD}bQb@YZIJ&Faa|>ZtK09-Y@nyzToWjYVQ`!l_pJ0qu`}p_K6w?V3Y!QR_I2%8-w}!* z$wbxR=iHFvOUm{pCiFl>69JAAE+73~+h}n>e%O9Au4B5K0j^6t%(j<&@V#@Xqg}gp z6(fUfM#)^nUsBJ^_?|!aGG|SnL@Kwb4O)~ut5j()=Fnl%<{G&_%vWxouwj4l{2eBHx{ zdc<-VdwDEam*#JY{sr09VNYYL8!#6zF3=pNPqa(2qLUwT z^%%{9Es9NYjYs4B5N#~J;@ZUEn^eCISIj;{Gks~`G>|9?#)Qn5Oz77^cuph78mEp; zZ`%Kmw*{>7aO!7f08{|90#}DpWGePJszAc1T@*L}YlvxD-IDzQU11Eqr1>AMM~P|& zI0xW}eY_`3WZK{vnm!ux+E~4p_|aZj8A%~eE-vP(VLIUN<$lMssTpHC|{tDAXkE%hB#%uGS$+te^uFKRXZ-WXBtn#yG z9o0UF$Ji3U1WbP&>Hx6Ou`c(sc=l-Wu&QM&LWt{SlByp6{g5aLWFayg^}x8b_3YMP zNy)A2%QkL=KT;@~FfOc%w|ee33cGh%cPGV*u|(YQd?ERYeujl!P z50PmpO{?TRCY~?0y)(q51CcE%fa_UsO(A5j=%Bh1s)N=zsA?U&8&OxMQ-D zXjYt#Xd_o9* z$W2vtXl=JWEmU>2Su*f%pdkRhcGKBL0_}LRm9*Im=|elVrJpRPbp|M)M$xh%L%~Lq zp<-FOjKp-R7NSG()VVeMRP_pwL9oZ}N(O*kp<<-KDaPiLy;1oMo~bC(czPS^+#=G8 z-JXYax7+=W`|7K@QYSD8L^%Fg(q)9Q;n8pR7TqY!n~#%*{-Qi)T>8_?UHj?>tuc3G zce2?fx&TkrqSLBIS0k@9&?{F-Sm-;oUJJUI&=N6lt3?tzhS#6jg%4L%)XquN30e$i z$QQWl-tf@T{Enhu6)Q{FwmZ^%BVj{S8k)?&Q1Sbc78%y2oT=?Oi| z7GJX@gFRr#oKlTb!3Sde4sD#{>_YZ_w@=#_Bfl9LPw-DMm#aUilC)Tl{@24F^m0Z~ zwZT2cZItMDf_Sp-t~GTJ9r6SgeRiG-I2_sYA`C13GVb0!6xu++dyd*C_-u4NBu6ro zBaf)lGdL>^P&9NJCp1KOVy(IeY4CY*`B!K=|AszFoHwgW_jOKzNQDMl3(+2G`t7Y& zfH=IM_(iTaU25826D9=QJI=TpnA|ZooMBt~=N9D9fcV{1SO|Xz$)Ah#})# z|8b;;F&yW1@f?a#-?(obJ2D6?f-pm!UVK5 z^atEF*d*Y^c#8q|S-qHQ?USB0ye#MW@W&_2fN4gF3#YJB3qt7<6LYk$gq50jAX+v0+t(!SjwSXs2y^}=GY%9t=zd^`vyY28(UG8*vJ@a%89sgRId!I;ymD%GhGP8Mzsk(~!@E>D~ zuxDBU@GwF8n3=ZmLs0N6DVU^|$hk z-`-F^OQ?vaH97qTN;;BP4wIEe{eo%+=UU|6Pkv%}dVXW+TdfIaV-xRvvDwq_-pSmV zy8I|QOSZs@+-f-=?onOGUwM4>;psbb-Gz=Vd&Y2pT4(%UOcd6zX)Y}E`|C4VTx#9; z4pVBX68Fo?xWwLi8zaNm=NsLU&AbtRXYUzOW*jOx;29pI<#LA?@AvwTE40>Fp({?Y zR9`VD@s7z%QIVkVS4M^lqdG&FUZlW%8bOZGL|L8=Ucx^$7>Z2907Y`uubGdzcVq)E ztG^b0EIga?v?65+kwPTbG7>VsVLa8YY}8=yQ$dMYnJv%wW|H~UV)m(0n{nD2L2X2t z#%j)?Z?=#UZ(p?`rZzEvob4P$+FMU0%caD%#Jnni*OOV(pSWyqyt-qBK!l<6v>GkKw!HHTaZt$;0J4?RuL&hlC4G(dm=0~V@r z8;#|Qf0q$27=Dj1)Xn?p6{{%ZYr^2!2MN0 zf>&WJBZ0zy=bjQx)+%9cW%#vDI8ZPQ%=~cKF)pI~iD)^G${XJ>gKteo+85r1%F7{xZke12L{r6xXkdLc9}T=0`_xKYegPS% z3y#XxQRWUmsA9N!P3w?il^M@NJ;zq1eH(t;Ky`RbU68k#Z4>TH_e6|I@G5N%T~=yY z*^%E;q_P0gR~~u(v2!rX>4rCLSftx#B|^0R8v>Ydz0IhI)3Lg!>#@`wqT!n2Dv@~2H3uh(79w_F)WpnZYFEb$>2GB{ z%P7B^h`^!hE{g=THz*M}{_?Fv1EHn|mJ2mH%L#gu_^0J$FZ<7c>c_*ScPsSM0Z@ec zRvq_Gh2cW@_348<6T8G>D#t4yot_f8JVyR&*XcHFWP*~7=VU+Y%_LIT+ z2ohWt{IZd)d2wT|mAY@S_O$^#ONNhDuJ^c5!U0i6vsXE;Bc{IyHfh+T$MCqt_Rg|{ z2Dka?)qYc>|8X1sKUVc4VAf?)xIn7R&}{B9$x1`g$h5Tp;H<0;?sXdvU;YW@b`Twcg3(c9{w-*98v5~+(|3|6q^}~}- z+~MirqKqyABB@u`rAPc}D1}DO-C;g~^iUSnKkS`8{bfJiIT72EVVl5sJ}o&VN8^qc z1=dJdKwXP`-pqU@__C9wD}hKi7&+Zu886fwJSj>sRe-Y5uzW77-h%O!P;C2l z6YVs7(fidTg;knLa-w?!B!q<$;8)89b06HSA2*x{=5P95025m*(N+HL_Pu8b5|A#M z+UNZwb_4z5qp@>Cd+^+gJ6-wnKYWj{h*M4JmY!Mc$`dg|8qJDU&9sM@A1aZT6<_8A ztp1Z207F5jM;3Bqa1`f4qtD>Lv$TW~DGzd$;J`h7O%`{SWcIuHDKuQ9-LpB|(D^%q zcqZB`iR-3fC2lAq58JCM@f(?4g0uQHaZG{xVgvmn(c%lAo<8fFF^po<{&SbNmyum- zwCXFfQ3`hJ6HHHuL?-$`(GQRd$0?~ml3rD~!&I0A|4>N^{Q2KZ2>5q2^TL{ip`7K+aer)t2|pTK7(8@^1r4 zDEQ#Muiua{kh+?7pm0seI(xPrE`VK$6 zt-a`L2|gyACC|?ZFYiRa>3cq^hc>s}phd601rvL0_uTr~hisW43FA zvp;1)k4yh_D9yRmxfxgn_AD7<$v;Df;TNxuoV&wO2^np)>bLVJ&v5Makb1&DO zrGGvLQ?kCYYZPyuYq9X;Dc^HY4vvhFvwvM_2~ZJ+c@+Vr>`8ore;Rk_j?ALh2<5 zz3f+C?|`QW<+GES{?XWV&b|%#u!k($`3uSu92U5L^N4b04nF+lWvw7CAS&?tFM@cQ zwXS(JwE{VYDRcyH1W8@knP9Ny&&~-4PP~ezz3!%z2fgfY$=0b}eV&%S^_RCrwpUqU z=(3Z>YtwO%*?F5a;1yzp;o}<5sSv}>?}c?x+Kl77aKOq)7rNOakH45-na@@iXY=6#r$kEh= znaA5?{`y=McvEA{m+jXvy@1PsD2k*fb1mg<$$-rD& z?YQ<5_j>zIW*3@G6ZVe@7*y1bIKKfREr&yM{@~)Xwp0LLwUL1V^U^&Cf2Z@OrPa|^ zcRDTyZU?CV#Q}R6ZLcg>t$NT=DE@q8;{tRP!0yO#klJ%O*HYlW`!B*TPJc~u9|!p~ zb1-Dqkpv?*1OZyA;T%?UNKxAN=bHoR-lek03pgM@oauyJcq)#g zel^OrYF~7tx79y-16t%?bl+MuJa$(fdTbmNU8^?uZ>Cb+e+-!Th@Bd?3T}^Gxm8q? zO~>}~>dlZ1eB(*((lOIngX&DSxlCAip@kc#(S`p|kItxntl4FY$o@sC6jf$pERCZu zmn7`#Ygh@O5-=yRC~WkP_6f)l%q?(it$}ID8+|qWl5M-&;a;qpw`?C?!<7|b z?hrYvioRrt8|#LpTD`V5I_Py}15BSu=2eE&tQOIN zmr(OrY!Nid=p^~#56p2Lb~`HEvBfFv5psFy;hXmSt#;$=$VN2Dk04ng{hB2j4rO4& zow{RM!)S`He?uu*N4^l{i{CaxjCVS$qckJ2`m8$;cCN2{?S#KFtQZ6igg=uGS|Nlw zS4KSJnjD?j1UXj-^mP~96hORwm|d^w%z3P9h@_tf$i5cwao>Y&CtJ%AX2-aUENvvY zzO9t!ALS!_o3MDH>pFV2+&~Fhr0Gp-1wQp>mjcpi#!KT`-1-wO-KvlB-+SluMtof@ z@;Op!`+jvX?DB(6;@|GJJD-co_hN?8B2 zCvolb@8=+}nYvYH_Gp&x-I}cJL*>c0C_22A4|m+FagGlkEdEOlDh#g-Zq$&hAQr|>rA{)-p700QevPWtcj zrJi|yT`$kob^xvlATCYN3;y9{z+Sy1K&{k->H&IQ3Pf6Rz_zWW+f)wU=#+sh&%V~26C`;5dAfR z8StO+GPU7uzXwwmBMlEowe{ak$3bs$uGU^^TfT7T%(nk7uU>HNxBRB=Fm^GC)XJTS zv8u3R5_TmMGxvSK8zi2nO)n-N%rd7zKHFbL4Fz{8&jkm7jV76c>|r9IpYpsEBoCOin)8N-lZ%V zehnns)jdW&hHI~SoR%aKOJQXH#0+-=?3dh|(%q(j#t79!MZK&tPwDpAr02}mbMb-d z1Ecf`=xPs6+ClH}qDvohGfv!^(i0~~(%Z^FZNx{-t}Waf|Aa^MUb}azgElU{w>>U1 zexulJa0>{y--WueZoSw?R_GTte{e2K&^NFjEMFKPPnQzYxVUbRlaIj%8k9`r7XZ8d`y=RegHzj|eLJ-&mtVk&Sq_kerv z(RTl2G{i5w@M2&fkKN#6M+#C|w*kfxSR!6X1@QJGtuV)3ij2;1OLBM&ZbatuyY9Vv z`ewRrEyN}8sSKNAmD(>P)4LG3jo7ypBvsj*8I+xLwFzvB=X3+^cs~PvEVV^GF4U1F zy^1%htTyuAKj&h(J(PYh$g^ir=&i4!CIO=il08AYVDcLx<3(D zm8=~Ylf33J+UWrre2x1B0v-!8ub(%KQ~9#{M1sjSuJ@$_9l6qWKtdL0ic*o=*$0lw zI8}zWL4I~V_kBEC?7(bSJ7}62pqAY6_}SaFmnXs~ER&2KbAwfmA>xcI)FqqvIkC^# zOFBm*pUxDof*XNW2-R$tQWXvJM;ZsNlEi#flmOOMtW%HaW+RVy*IZsV&6H+?Jwr*) zVvw0A36E9!7{z28z+5ohiP`Qf?u#^X&xdpK!Ni86?yuEaVw3j32j*=HzleY^&XdBI zux&8BvWj)cSPHupjB}mpS#&+MAi!{ATaL?;7%HCM5$c+wJ~_g2~HhIob zLnxLe5>`o;>lhB(5+= zb)0S;JbPG@4Go+`Y)Vr3U>foK?6c#SXgZ_UJmZ3=CNARvi-E^7JCDP^(MhHd{w66J ztHhu|;CZ0S2qNA?3s}cF1^+Mu-A^8q?tOC1K))Zy)`;1fi9`ra=yE;U%tKECvwQC) zZ>te@gI+ZEUXKokPDK;pH2V-CA&{xCLX5qH8)DyxlFJ+%ui0WOb^D+unb6?_k9f6l zGF)`vTCl7{j^vs&XE-k$^BAkSOp-;Tq~e4Cf(!RGdFU5jj7mpI424R1=SZC+kMcgx8tfxEZQ-`6cm+3Wc2UzWP#<{L2F(ZCku8;@z* z@DNdtlIxY_WN?lCnHG#p!gL{r@M5^!lVa(Xgrf`|6s(qZ`*Rbs#0}l3$eA+8J8^%w zc3F<}8z%1HC1RP{+93`3zaWQY3}^-_4I^G+hYVf4-Y@hW02b@^&V^JXh$z>7&<$R- zEPI~^HSDi@&Ydsrf(67k%AUo#8gh;wggrA-uV~NHo1Y}Gqp)txn&z5_5g608SNAy@ zfk^&yG##<&1=6f*U>Q!(-K`_+MsUS3@~N+0?BAWS_52WAdYG$^xueVpAq=i>fF+@# zd+m<6m#Hrw!5>Q%4YJogox)3k&@Q zWRm9w?dh&#yW%?WKRs85YuNSGyN~tRvGVm6+(V)p3nK*KarJaiFy@@2iNgw|o%dUM z&Ab0egEi=a?OB3a*DpEcC|rdM_FenNe|`|{iPbWY&=}Ia5DE4N^K3mziQ{yP+$ugg zlUQ(mMbV7^`CJ=iAa=SpC6cPLdUsW`0a)f(J}p6-@FreH^rUm{kZK0lkW-hmb(3{u z72h}Z+voNk0lx_zsypou@;n>JTvn&f@K}B4`#YQTjU*0RH>Z+uWe-)-)-7jjrTx@k zQ1wRUlWvwQIy&Fq#_ilR2SRYnhmt+g22(|Gg(H<=IUnvGJHA(%@xBolw-8 zkY~}oFxOERS{KZMOrx5l4bH|g+>bE||F>BVPp=E-y=2gS`t>ri#}bJh;_=lO(Xe<_ z4wq~}{__C=&5Ju)Cm;dtR^E`z`{5hQ>1xY)2m(*!X@@ml+_zfj<;Bw`#qGtjs5wK< z{X@Q}VW&0H#%g2I>4_i|?eA2tbD%MjS4L+23==Xw;^w$E9f^$F>8G#yzeG^7vC4!qV@E>dCBVG8LiS+{>g(v@L7NQ({~1~`j&{^hE>^( zD;Psm8i;`3q(bqMWS{E?&BAcvIj~8$9{67FX^uofn4ps)rA7h9ysiO zx#O&wxC6DJ_+^=|H85E2rLO?Q1s2|=%cH~$y6a~X*7I2kbMPO%XZ_uBcpu%0Y!!Yl zLK@Q(SnaDHe}?xeEpJAXsG8DE}M1sd(6Anu$8c>pO#AgHVP;nQl1YRl^jIVGH%)kupfK>3bk3cio8vTy-WAw zC6#8rbt)s}RNQxuxW8#F&|UX%E&mfr0OJ<7b?;+9Vd9h3q>eyHyM!vf)$UkCBYzga>4Ki!D+#t_Ewt@N@fU7j08zSSt8NWu+}AvQDz{&h>= z`A;7k8d76KOW}NrU*E>IniE62#>iiabrW8n4Fs{sHr3+H3=LK`tKj9%!AyNG>yWr9 zKj@?W%G|e*<9Pd@@<+TKD}cXbKUbzte}C0)vJ&BT6gZb9k+(-@=Qj7~S3nuruytYQ zlHA8WLj(J9@|ZJtvzwzElQisJ=pa0yo45V@fo*%5k%$gKuIEjiOUt4Wu|Zl zSTKS-nYR4p?WhB7s81}gmB0H+;Itv#webXMoN+&|sjLk+PnnLhoVBb`^Zu9wgSrkwhs`4diJk=Ls`_W%uW0-0} zdLF|9vUiXvLitYXZe!XKO8WG_s!&_W+BVQ zq3y*NAjWkjRKm{v<@Oj&MkfI8{t%-!fM)8~ERc%LhQ&pb7(h-}8vo zLiz>tizJ=bO1cb>Lg;Z=+=MO5_@<-#vP`}w4Adb;V$Ymw8V0U1w>$ZEi~U)&ekKtg z`j_>K%gXiV*<^5>o1=sSV9EnQ**AWdhX5X#Z+|*xd#L83 zBzoWwKgT|B%}3cb-qCF)t;!?TfsE@U%7xulZY~TJjysW8F=6NmuJD1LDF_{;dSNNb zv3-1rH8DaBByH-Hk34yVxzKTw%C_h4oj}tvocJ7MqtqePC_Q>!Me}wA6>Rz?L>M=H z^#+1J_Vz1^V>;u2^bT^!$6tigYC5K<8aU@Q5)Vy+7&ja@N~ssIQt${cdU<|iO8XzT zsk~NYKM+(PwYe?~0axFTEqgz52LPBWf91R*^(zobR|x0ZP^tqxXN|CXZq1EPT zwDjnyqX6HrPyS4IllG&X@UJY()OQtNgV_#M_j!hQ7~zg#5@%mxoG6a|t#JDt$|Y0r z##57Lsba7DPl<*PI$#w7Zb0%qN>*%ev&f?K1kwk{0(gz-Vv*M^NyF~KR#dY7Xcbx0 zIUS{T1B-h2OxVi~VS8CeQ{9~xqeO3xHC+?mKPjl7x3*sce~yD=NQ_x0RQu0VM>B+Y zrh|_WTP;R!g3(x=LtNH68G=yGBnHQWxtASb5>?3EF0+jwLSj>qp{yRB0-hl2my+)7 z60YD|ku$)tWY&pB7Eu6m985wLdOdlEerDRM3LiC>5Q=k6&B`L^Ephe%tJ6{?sNmRW zK@!$WMwKfNw&8sqDNH3bnGD*y_X732Lw`a3QEZ|CP+H}-tL^fg;qKMw;oJ&1%%kJ( zPGCege|fL=4L)&DZ9#3wD#rz3Ip>z}u`Vn1-~RcG;Q&E+2-gZcN6bbQ$+>Knszq%c z34iwap@z(-)bX-tZ`QCtBoZyaG5xQ*Y}W-n2EX9il2 zU)#+F+&55r=7&Zp6!k}ng@6#Hbhp1iG@*IESER$H! z2~XV=T-&h_C&Z^_x{#g7W{Z#5>=9IPY)>^*(?|Sec~m%nh432d;zg{uF>p)uN1hb^ zyT^gYrB!a&$_9!bKSb;cWza1Cv}+tuJfMg0_dg#}ffJq8?-F1cuvN&4By;6Rs&dZk zuz68ekLaAXr&_)RYA!cAHSloKJ?E?i{h6pL&4FDE0e%`ecwXd z4?RD|17R)3mS%d%@H9#%D}jYWMAydWPc}djfEyKjiYgY`#^lgzr-)nb6NjJp3tlRA z0yH+>l+3wlv*o|FTHm6_F^A=^KkbVQKEaI=3lA9>AyTN z9O0i_ahO~#b^Z(HkkQSa61jTTl@6>**!EyD8jtW@%^GuzeKDnV?V>lamfbjL zC}Sk>ia7A6C(9)KEMV3tSPa*_vZ+S_!!?_wLJn}2)u`*dw;`reWe(whz&fGUZ0N?e z^IUAqDDz{5F|^ap4iksuBz$2;NG+H3T`yYav89FdQM-`g;1yPUfruY->&oggPPkm9 zw86CN4Ra&2cL(qw{*m}1HqIu7n`->1^%rjxYjQ#{;|8O zWD{C+n$A}Nq&Il;>*4ipBAR|AJ?Y^X$}|C(eOKTtH6Dc`+t%m(QH=p@g?lHKl=91V z;D)|M49mi&^WlR1+7dO9I=_2Aia;-G$BkhdADIK9oz0tXDBU3|ocmv*u9*Mu<64hR~XTZzuPAFamC%ltsS6@I!IW5Ec}ZIUY!*{1%>IM5@Gt zD?=zXl5;viLXwxqC9Wdi^?IxWpLSlO#!kh(EjuiR?q8?F#AWwuA!M+k`}3V+vvCKP z)6X27gBS5z(ZKcg=VTLy9RK$=h^j!ON2_F)K$Stb>H|23#0|{a0qg!t^PjaXf1XiTjJ-c&gv+r!YE|eP=@>R4DYV3Zll+6s$T^n7yR;*!ps)xN ze)UANehL~Ju@*tcEP^puuGAwZXp)iynDeToed1Xic2(o zJu7&cdI!ViSP@(iO-r27$Y1Gtb7M~y%(8QY(tLm$Bnbst#uv4#j_7S9j_~V=JJJRZ z@fiD_cBSJO(%Z6Zf!%1ptndOlf5osUjbAM-aw`p+KV+NprAh)jqj+S1sci&lI+7jT1#Zoz?_XvRn9xK@tr7P-zAtXT|-lV1$1&ZnLcH%nE=kB(JFOD%G;(3Ia!2v@M16HtvtAp?3 zJkJxI`bhtsg`H*x&6Yl1ze!vqXD?+HJpPv+$485v^(J`dP*W{p3Dy+(5-Iomj~2Bb zuly%@Ca#r8oN^VB5KOa6abT1qFA9Tu!J^F!u_8teBekotl!7w@h@6A?V4iLDw-q9U zEQ*H8s$1Dje^;nN-xaJ;Bb&uhjHDj~F0!_&>J1rbebrreX`q6=q8ehuLaEO}!9>$o zDf)@ZSpeHTw`l$7aqsGUFQ?^rvS} zIw^LG2D1>cF@}l9+;@M5HF>fAeY+5?GaGlIXvlcpP1CQMjN*BHAaG+}{H!M2o69cz zJA<2%jDRsB#YX~tV^*%J6dRwTwm>n^BE?{Gq7!+$c|W0aux3_~M)_`NEfVKz9QUJY zUvSaYoh{k6_tPS$DrDTv(-M0*mXG5M4$so9T<41DzZ#A1KL)><-JUFL9#od;ET{F&HHU4B97x*_+ z6zh&fsfXvlk0{$VG3A`w5q?TRa3ps5HDKT}YyMOLxl#V|9v=5hm=wlu zyO;+9-ePi4BGFV8?V@7U$HYMN+M5PIv!OUgx?mQD`58e29vcq7~Bm2uC!y~9GRFl6Nd>l!@P_4?o z1~5d*@=r{GP9bHDA0m6a`ZRz0Vt1~Jve7jEmlWVAL%@H8-#f1+{erc*>b3;;l+s&3 zqGO;nPuk5@rMR5p8ioVO3E>AO{k%y;N_^0te5mr(*l78=Cll8)8^0IU1?S$<7s2{D zh)(trVPRv0fmeDR^GRm^Z|L?(iy7xgFWlcKcTqU>922!Agln-3ROk44#d<+=BYW{xnq4hc>9O4XFUMoBV?9<++~n|c-vgI%q0!KF$Lkt;sp zZh9YBuGcTz+aY#79AG zO=5+9JNu0+%PGW5@Qe^f9={mp)f5ke?6h+a5+ZbTSr7f}4MD(50ehRh1wyF^p!Z>~ z7yl3n+EsGc!YEod!N5qw_mldfRCwh%6v)DdDsN-%! zHRoeQ;?kx)SCC!9{}8iB3q~0H0Idr-Z7)354ae(|2~LR$?5=Vjd~4*TsG=k$k;Ifl zmIj}HW{%izLj{53q;E?>o}4hSXY^NHjoMsW;WV;57B^kfpOgY&`%N2Z_c$0+PD~Lg zAA5E1diaRQ4-8t?3~lZ9U;nq&%=K4vh4_;faXdgd6%5S&%O9HX?{9ihbTH^ogq3^B>|z29QP z)Qzqt@zknjAhK{D>YMN!U5_w=rdmEwz$ytNAFYbLRJ8BecH>J;G*{l{v|j3?2U4l= zZ{dT2jG)p7_aN=^$3+dmzy+Md1P}J2*cKC$`PMqdnkneWH40)hxZ!6mDcPFnsu(=F(>Ub)W{9T}gbcfX!c68l~n{~}KlWOrq9 zMOsg_&FAc8{ud;2bw7dytbGen%mqvD|1yIY!=zV}Up95l&6xPD_Dtfk=%>p)YXmVY zYr1`jQh9y(v4C}rvKD);1arVAVM_A9qguL0b(yGkC^yIFT(g?}o+-*_*HYC873-T` z4eaGC>QIXp(Z{rrUBt-FvgKgw(?h665YAIbe36lIE)E*P6IyBsPtO zf0h2vX%zUw8$>2^k58}GSxkr$tp7p$kIO*b8zyIa)tc@*GmL%v!bj5cwOtm&kNBJ! z(j1N`b*lFTP+wfu(W1^ZUNfjiQOTzy&|}8$OuP>w1Hwh%FFA68=0w0;5Jtki#yIZBbrVH_~|z}3g}-)wDXzxxY>Wk7%7&9i~DXI*<*eHw0J-wvmp@jIhBjQf#$y-qNU z%7U&q;~u<{_r+IrWf*)8AL3h_SmFK z&AJ`GIfLWzw%J^OW$GDL^bMdZALH*cVn1}*WJ^>1L6G43TjY;!`-)uj@%lTAIqAF1 zeX1^_IO{4SU2$YiPvmaK-_OlJGt{luH>a^lI&19VVfaKPn40niUv&d~Ni3yT&p*DJ z`-@e|&FeRhVt3+1fQ&{Jcgc zC~i9X9qJa9Gq0P;_96c5ZpTEEN0`^LIUx>Z@hXKFmCDJoi;**l2a+@xCoY=$dL&xB z-zA382~n&Jes}9mO{s>%(x2W!;pkkpkkUK(!cjyf{hf#<8Hc7rUe-6p|B2qot=zW2 zCE-rynE=_~Y#qi!2}{pWGe)_J_lDFLkBk!t1}~GJiF=Ua?Ln7VEli5N3QF2aqK!LT zTS$8a+GvF?>`g7rw|&E=mJ`Eh7hiWlGqz?eN@EC$A*EI=<9*KvP=~^rvL4-_!V$2! z-sfdUi)mX!&G(Kp(W@PHqmmOw*zJzf5V8<)0AJ6+%_Boi&61Ozw3ue069m+!AFG&q zX$)_u{HK|4#4ycKL{YS9@!AZ2ZGg0Z??QJ?)W_(^Sl%xTXS)x>wC7rd#qBrz!ph0y(fjum8C-e@co7Y zZ>4pb`oPm|dY*-iQ?u2?g-K0;8bUM)=se_I8T6=~Sp+L>3Tbz9hC*^iY?u2e!rleIHtKxcJ=yt&JxBqpew^uO6Aig5#szZPiand ziw;8~s-Zjj6cQotdPDj3@AWD!m|_bZO)t)&l$-~@_GA?=_WAraiLS4im&i-h%}GNG zYwR%b;elYmErCjPew3>8kvB;K-m-5cJarPl=ZMVER2`s;gigfSKjqQ>uBXmh{MeYv znPDF_%o=EK(52OdH#S>n0nQBWpexv z0Bk^$zlDO&?f*Tcmpgm>F@S5=t|h3Lz-z}gKU!6*6;cl7mi4>8!6n-Up+$Ac*?0etVm)i{+2 zf*e7RA)nigU+(cNAU{0pcY4m9jd%v=GK|v%?P1ESP#RC>%t)Qd1}S1^T~-i{8G5QZ ze=VfP8b^7H$o%nm{^l;eG+Ic>LRDQAY0_1IX1I4l{o_C9-3vIs=Xwf$+fHO(!UaFD zvUj<{Y&#bLorDp78ERR{#}@aYiOAnlCE&=2KkYu@DXVyNZUH{)`>4~jw^-9+K1vdp#RpDia7vHf>3-q~))fejh!k z+Y(YLgvTX}Lp)9x!0jIhU100py9l`e%9#MvZ{>17`L=D8%Z~y#0%u`d2%HZrPGbb= zs}eOD@s5i4qxx9NwH1tl-2r?X_?elzTMCA@?K}xE2ZQ zFeMnkmoHpLIqPU2^(mD^d(LkQ%$p^2hJ)uixLRV72v@_CR8#bmV!x>5%8D2OV|IYV z8>dYJZOsIl0O9o(D~n32^=U%U-)hYOfZGGJKf>%*VC0p@kP~pSv35h~87wV4 ziQJ|;!fB4BjFj;gI5@{-7n`x3FtTt4MhDOi));!-!bo_e#9!&jQH&vs0|xga`<`#X zkC(TdvSs0;r_wn%hkVdZ;7SZxC`b$V?DZQayIGG2Q%VIWBDH0Lb~Byru-{SWKc0tF z%Yj!V`XuX8WUCUtP>kSHxm~2No}s7y(KOA~*BGJ;DJnuUW55``{#JF1&NyfR<*C8G zCl(6LDtibeoEt1O60Smap0`KR! zLg&V=;_Sn-3JZ%X_Meq0XG)lv1qxz({qcxN{e-@HZbjsG@SWG4%$1+rw8{m%MACC4 z);X9ZP?hLoqkcsEukKUZj}w1QMX3FsaPtu#al_;E5Z&mxMi*ccHK0;{B;tO;n-wkl zWf16^Qo-%X=7fvYF{^_#Xdpm{FlRPMuEbqijVZERpgIO|G0;>%>$&AH> zi+9e!tkbB>3~VA5R3$A;=9w#J{QPKJsk6{t>K<(`_Kb8DXAicOx(?_4_C5Jh$Ig7Y zV-Mzrji6uzBPQz$jQuFsZ?3#|by0nfI#*j`E$m;PvOET}9kltyDJvOF3Iq7eWh+>J z{pH+!{;zOlhjTU;bcxV&9b6)GiWC}EVYXt(Y%&l=9_{Vq2-J;4EWc1Xy4MWw4?dMX+M%1+W1#>o=?)=)>$V z*j~eS8@7wU-+kGk(!T3^bCui?0YOkAuo9BTf8F%q$!=a?5*Wbe&s;;0DbqIENqN?g z=MCjoIPnbNkhyMPHrPUgg&2#Dvu-hRSWg%fTBw7^h1?XK*ej`VimEQ8rc z#fc)Le9Y$`H3HVFraw>JcGol-kI1=Jbz;wD%;K zzVFQhPD88%P6JL39Lt2d6nckq7Uo~b%E(rGrP?>uHZ!Umq55Iei98ya8P40NL~}@$ z^NQ538huq^0*V-hqo(2Tvhel-;bE4QI0Msa6E765&#=2N`zi2?FW>U9hi=_^qXE3^ zwcnbQc0VB!z-P{0gOeC3`*i0rb1bkIVS1IM^Br^~4R@~gw_X;G`Uz1#wNBh#5K{wc z#MC-YztO|LSBAmoE7nqGP3+M&f)01^1N*hvdS%XSd9l>LFNEPdo$5APv zY6FU>3f6e9oztly_gLrKpHsi4B*pe ztqD1@3p_7y%N*`X!K(!4M@9EB89?Q@t{!NN>ymbYh<7N>H&2}SQW-!<)T0JNh7DOI z1+^uKJ#Gxlm|2c6RaTclw^ectG*1Z9lmYz0q>lpM1j;YpI5j;08(#2e2p{ilm!kqk4J#NFF$OKv!}ptUz+?x3;aI|8V&5zQ z)@EiMz#I$JopY-n`B5-{Zz25?;nG?D%ccke_>*hGkF=cg={}gR8P>YGW@%)wuPac{ z1J*0E`G!t2bQWU0!W;oD*WhcKrYqtxi51q`F;4JA`n`7FSd?ZnNuZI~NgLT~Qb46v zeZ;R_XJ!q|!YshTH0Hm>>>GmX&1@Ti9S(%=0@d`?&)hQc%;31&0J;|k4DNI%fnEl@ zjZnOcbc||y5a&p-vg(N3+@K$pN0h!3+HgO zEBx-N*_UV)pxw-bYpUw<)y}T!^7T0vs+8Ak1gaCVP{jUYk>j{cfZF-$*Q!YRhG?D2 zG10HObw@>>zYTb@!I=T}QsDE59|hP}hkTA>e;*eH@QE`{A*2#xV4FJyr{6)mP9gCh zyVMc2#>HmlR64s-9%BZHSkcHxy%o3<;|>}3gFS}SZ(-W45u{-s6=n|aE)C>ofsTA( z$B;`{S;iuZjd+dy+N!4w4h3bmOXEwjecbWh=X@l5sv#kBT>AZ0{NgzeF9psw>>>;N zvw=mxOi+(R`}ptP_?bkl_esVOCHN|#hJJvWWiBLhMTpf@VI+0qUy=!g30^7#PjY&9 z?NL>&S~D_Q^@nQLRgH%h{4g^S}Mb zg!bWtGk{N=xf;yy1Lq`LX5!U?R{%?*jdnC#z*4TO5qqLjD&0?%&sNSAfkzDP2JSa_ z6!aLdEmLOq;Tdhi`Oy;252(ibI>F)2MPxb`V#R)*e*N|1e&%nSzm70b86ChWmA}nj zusO=WfjLY2`u9HCd+Z%`6xi^bkKhqd#1T|TtY{-%miFZ)JL`t$JhIN=RwGUVog~?! z@U;{ltr>s+N1LQ3E2h|UY9ZV)Mo9&-KYM+Ri%gOv&HE->TOVj z+7n-DM9Qd~Z39*q%tb6Nd;Tfs?R@%oz`w&xh1Z|)$5>?y`s^M5a?E`?p$yAn zz{~7cP@Xdn(={euE?Cv7gPV%;DK(O=I5oLTT>n#<(0%q8unFvT#BTzhyPf9-tDPD+ zkXB#QoEF zXL7{1h8NI=;@l{z0dwvYU7d|uHd@>+B~d#SZ<-v#lW)&m+_uY zIA{y*l}6SS$Mg7K>J{*W)-`mv+# z=LwYn{9+bAZ+TbdVuN=HmR5r@>J3+a>10{_tj1NRx?45qguiEiMgrI<<7=SnCAI@Y zUdb=~>=_qZ*YGg@2?&FbL_6Sn(SnY!o!T|2P?)d=BUa^Dr?!ik9 zy-{KQ{amC+D$!K+caO^RRw)P#GsILqi!j>S03V_Ms=i<8wo$?D$i4;ojllz>BW`bJ zM-Tw`S;wZ6mh+ZR{`UzWf2!-#&oZ%+%9Q4``2yp((+*(Y@q1F^zh3i4{C4wtW_2FI z08f4{u*|7Qmzt-GBu)b@CF04c5BiBDY-&yLscfOC>AY^?k6jTRTr7HNRi}^If@myj z!USsHS4>cksvC)Dz?c+J|9uiFM3a7vGXkYuAOu+x%Vc(ynaw0zeB?KV4ZZTb-v{8! zxBmMP_xdq2fRCNA8n~ZaN15VCW|f(HfsCgiauySA(or+jG?V?6+>J^HRlTj&srO_0 zrJR0ZD&LqbckL>>QS$kKvLDue@-274EqC$OV}0>4aP37Gg*)XzOw1vSbc?tU5?%>Q z{qCa&+sxRf3FTVh4}9fKZ)W3TA7jh@<(xEU2Zu)5=Ymf6Eqk`m*@$z2Q$e}t)luS) zlpX=`Hz}<8@fyr(jTc1|tk#dKZagZXyh$byS%^KR$JY&mQ6^CH1TjS4l053oP_xZ{#N5%w>i2-a_yOw|g85gKH zobAE5+Ta<43vV_=k~|ErHfM;$Xdio*dN0@A zmT98jcv{9lJjk(u5%w6ae`4`DZSb z+46c@$BBHbzBecs_7KoNq47WAH{^kx-Ig7_Qenc?)xd0_wrZUkX)7TC)OOQ+{B^=_ zB=|_}BUESVuWtwiz)oO`$#!7umT4ET%VfI^9S{`2%0j+j;WF*Qm*HFUD9)M$Kv0$_1DK?dzXk=>cQ^^tawBg1OTKJ#hF}H4fTccwR~$tW z(Dy#`pAkl&aD2+jaz$gUiDw8d7IrqU6y!w>Y-z+FBWTqpTqni+B;lt?15HeUjV*u; z0y`ykVC=BaCAv$n*We(iU(l~m6&xsd{yL1<^)69e;fS9j%?d)^V7(cCU`D|Yg6lnJLoYk`O#poL zwokRZ^HDJX3&QSNx9$<{xqsDJz%_)ET@oUErTa|rcZYg-Wvzom;!nftR>}&3I}Q6X zf&FhsNBg#e{TUE=4qmYFHYOY2eDWHs1VP&Ge1Qxmzd?JP#(X(V>S=A&th4jpX;uB=`r;uz+XJ$ z@0n#o1Zc>@F!3E`ko8LRbZ_O*q1E$^V3mbyAD%%dO)U#Ibu{*Ryd_ctgA%op?#Br7 zz|)ZJ^BluD8@XX;8GlwL%L~tb9pAX)vrXUYh#9~K*Q_KAtb;r5IeDSy*(J!Hrph|- z={kN?qmRE_ozUZ^kh+^MrV^AfyHig8X7DXM9d7SvK+hfr$anZX zgOcw$faPFwo4NVYrUTd827o>%Z5|vwG%~k){lTaD4JQR)9VUG zgOL;7k4FU=1esA3+z)KVy#EFNeBf%qvw;}~UOW~M(x_#8s!ME0&6@Ed!<9_IhCQSE z4tmP895s%d0T_%93_D0=U+L zZ)Gy{m&@U8F8bk($D*UJkz7cM| zwdwD^>vk@syKBMhCB!^lacXNedc%aSfzf-%q;w#vuNthQgYJ z0EUJH_k(T3xC8Mp;XI843hc1jdq=D^L};@M?>XcTA;E@kCr0|OO-}zt-|>~7gK(L^ z5O2TieIVa}5>OiH+BMKNu;8%dZ$kD{Gp-?A6m>?ld77F8V(|fu6;m#oD|ouYJ(P3# z&jEceIp>Xh?e_m_di!zXgKJh|a+t^qUMzTr6YFa$@>D*M*nXwsqlrG_i8@lQiIh9= z5xX7wxX_mbo2w7~!i~p2CjLnAz16EJ^|pc31-u6155P{3j`GpOqP4PEdF|{Y2A>5! z0`xEFnV0I;{PSo0V{N}wXeZk-I>$_xICOca=j#-xM(S7n=5_O3nko=hPFnk0kNirt z0SH7t=n-Lef*vNEkoA!7>(Ok+`h#lY1mG7A(|^y$>0a}y6Ot0^c;RiAz8?T*4&NK4 zSoCOXAMGiX-9={hT;K}BPKzfLNhS@6WD{ykB2rH?SGy*t)*ChKHn8`|{F@C%{ffB! z^}D_hd(I=40EWA|=olQFW#THq>X?mAvUM(>Wf@uA6f@wGtzNwXPQHiM2$2Yc~2 z)51wJi3q<&;*T5AH@5IPkg4 zJJs>t%939{R-`s*`!R-*YAEayHX}pk;0+hwYv(++j<^5trA@p8Pdt3|XMb1Q|J_U8 z%g1kg-=^Pv%6tA3vj;)%1=dlWCf-UNXn_<~(GI-Q;GSZkZ?jx(2x#yM9MOvAKH$sE z573ohu@5^sibciQn0G7T>8g*27OPXc-nbX>3Bw*447^g#JMwHiCmMb}L;ItLd(6`r zN#eHz@kiUDq0A_nu>-U1ZKDNWDOO>l8d|7%h&LhqKj120CL)tcU=0LbWdjLgUHvpM zMiH|Eq!YiQb;5t$NNdZUf9}b9PU;$L8=mZLnpFJmrSBo{jT{1gf$`U%o3QF~?@e5$ z8XsqfTj_U;ILhV1;2kvAH_6(jtrUK6o~ zTdfuc4Sogoqd*L0@hJzvq=oj52iLA$8(KCUvw;gN%mN&1qiVa<{-%rm0NV^66_kcL zI%+W!@3{Q$Da$C!pp?tb8$R&lvb^_zUJslL)FUtKZ4_F&C6ggC+4src6vhuIs z7c#yq#~oEo;+l>3pQwI1Dg^uXan_RgnAyp|8wdk{vQ3GfbkV=U7k>+UFL3T51TVky zel|R>$VjoH(q_SeAgjxfUJJZL(lUY9VAM;B@UiGUr4TiV>r&1^qn)7U`fJPuP$|hu znkl%wy_|ildt`=x=@~EQvbEQ6{e9O>cGsOy+`9FboV)Z=1_ts-D&hxQ6|Qc35-sy) zjz>3WkS*IQp#|F|Jt9+Z>b$dfVDFv4I3$3-TfKt6TfL&9%ezxey2QXGNBP?SQTJBF z+8rYTi(VJ4Z&elfuo*wj1a^a!4SeHeQ=Iq#UU|+sh7XuCV<~WkL0etykED{0v;(9P zKfr@na9`M1##3&DLODm-_dE!eO1jF#yMR|H6!l0Q)f0QkK_to^>VMN+gN=V9^;;7^ zKt`|(@$6D2I5iU#81-_DdJ{9d@$rWL{Pp_@=Pvn>VNl{n$i6@r2z3-zfL5^6L01zn z!|RBf{l>2XgdKXanVnB9v4thX`0DQ^Mfye+p|0yETwz$jupb!wX3*2X3F?mtft+aT z)k%c0**#c&l4{Zo*n;>C7&Wo!wpTx}*<}upal5$T(jm@Ta3`MUuLOP<>0Q8iR#Og= zebr3=Z=z#H_iFz|B{I=isZs2&7S!zuRR$q|jL_A>UA(t@UZ$fwN=JDLyL3(<3W2{B zzq8{P9AwfOF04wDLePmYI1>72xT!dI2Qg=HYY|@GQW^7Tr|*L!j~}?$Zwo zW9#-QaeWvov`^wg&f`~}QLKs@U)yuyvtf-F-#C9AFJC+_jA%rc057$0wHrYr@l!i) zoH}Zut@R$@n~QH>@Zfz*-oj>{!rIxlv%kFX0*T*~cnQ){fs5(e5z$?-k-4?_ufzy0 zMTuKQz}0FbjebcLBIYtoG6)IOjTmV!7j}=7M<45uc;@LZ=EwK{?__u53CC^Qe#vdy ze#!b}mmoRJxgp>XW=|rvenu5Xr2>+UzB83QTWi=U zRC+ZLuO9wRq9D1>zvPw3(m?1RkC5aNh#U!XQVQB2OQqHkwWR%`E&+DE!dbyeXF4&6*Cz-gUS+DgI zMr+3jH})r_)&byp&?kVeg9>T)iI{mJztoKQV^=;|@Z1@he2?dOLSsGu6C3~e%fF}W z<(T!zFNX~M7VMWyRQk6%?actLKrGCRwqsrxA>5Jy{N?JUp;ta+`)3=hio5UP#0-rh zKCyazQv#n@zdoE?Zn)91on&EP$4nxOGEVr_R;rXW z+CxeP8&Ria@%<=6h?mDL;xU(D*~~UO&u&<> zYPdg>3wZU-n1$IxceAWI(W?s-91UlD@ zzXF~ui^Pvdq14@^FR;lQ2%res4Z6Xk4+NRu&vU2m`+C;XW{JB&zmhl@y-!r#i%S$O z>;iRF0ELUBycSqE(ihI!|HrGpGTGfZY52?y?;~S|hsvID6X=LPerrYkAu^br%`!{?6D8V8}0e0k*(kMa9RQ`ZKW~w{&I@JsC9uRzChm&G@D7)A!gz9p+`nH}!S0 z(P1UJtw7OS&de?`Sd%pLRoll}5P#*U-{2NxdxO0BytY)j2g?Ng@ zm7TqA(S#emABRxrYYUc0dkm{il4^p)movWh%Mh;0y`t*YZ5awIdAKmyz-iT}w&>4S z<9g$CYfSJJvhm>p2QC@H8bf71#TM|)z zF)1k0o`=lr7rAob;0^hAKHa1Ay9yI?c8&X?nzeBw0$>Z`>%x8ty1ndm?)DhP6-973 zPiF2g@`Dkw?pDw(z)D~i$uZv%Qz@mjV+VGs!Qw7+47hQq019K2*II2=o2c8)QsdZL ziNI+~VjJe0*~88ScgqDn*yBR{f6A4Y;bk)ff%O<%2`q1nk7z~wk-y=8dwpep^V`|4 zJjYq-4PehCO8iP4!Q)3`sQi`vL%`2~e@FI7x<-G9YX0;CuvTM$2la}~}Kyfowa7aZC#gV`(hGkfLK_4pi5eDc>Hq-<85 zQeVxARL8i5q^1>R&~31!&&s_Wfi;%^{$$CL>Wy;1$(X{CO(d4!u8;JwM=%-|^Qa4n zl`F%10cNlrc(~|$eHr)o5d8smZrRC0yEg0eC99W#t^npVu1cTo1&Ov^rC`X%+xl?V zQ~jdX{#s$r!RV>e)L6vNn8Z)$Vb}{?Z}64uK=1cQ7jG@+`etF7G8=yI$Fa(P^}V<9 z*H?WSibmivzsUH(*}(a#WVT6KM$Ye8|zU<`1NKXM<7GRkMX{W)^7uk#2La;}MGPAY~d9z+A z>}gc;@gF&wMvxSN$4q>PP<#2lWefg{ht^TG2)BUcOa39?M&M_la&z}zJeVkr_-kgs zdaOv7;8{W7UK{wCuH;hg|CdnvCQLc2EyPm&W+LK!)Q24TlX}m_qq&HMsD%x{402h= zXd&p3SSnc?M3(l7NCB}QvCx=dwW+@GslwsIM;*^*m>v*%6yQyFPJPk;%jc}EHxir- zTxrnJ7zdHuPx$^LXUl3kxIeqf59F=^7B+SWP3U@D7zJ)Y{43};CU#SqeHLVk$i&}% z=MPgato~$>0Vn?!0`Gr;Cxd1~D%F$14kAY>sz4|ztP;G?;1SGy7t!~jYp&%T6OhpJ z@q~J7S(~+5N}eV(pJoC`R5olT7S1_n$^ddX*xx^#>CETn2zA!S@`zzCYH+Hi{3gz; zs4#(0p^Uv@wT^*3;yJeaj|0a4naft-kCwuxF6L%Ag7bj0i6ljBZ1&G;-tgpi zhT~y4lFSjzYa|IZNYe4MtrUXV7uQQ=I)76YKu)yfbDfBVfY&UVFIJv3_*J)r;VTj> zWI&%`JBLTxhF$wXraE5#^DXq=wVWCEEk&}6fTt50NY#?~6I~E3E4YrnzRdi&CFRef zWmufGIBF;jEy>t-Fg|`#&q?3RxXeH%lLy56zyG!)dhg-BJlXaVCbl8I0&G#@m6g~@ z^*kntKR!R9UDBmOukmmfnOu9uglFO-*Zj{h_1ohH63S(rULf@ zaKRc5t$WNJK5bhs*s}~SjlX;1Urnb=1_af$xx?PVwn1-pr@}SpjtEVdfB}2W>{~v3 zWZux;pWB09Ww_8!$uID}@BQJC{Rr@`>)wjL>>%D?+W?v1*T`-J+Y_E`5{cB{B23ne z1P;|>1TsP|5qga?I}gZbg0SwtzU+xRU*L6@ZKx8CqvX#AEhpjr9a})IkF=I{uD+MS zEQeVOH(&q?=rFVShI!WHcJUED>LQE<`82x!!sD{Rc7vhEf%Ff6${wzi$-7eKIc9c& zVV!Z3r-A5WZ9;@mm|1`;^k?Vyj(9VA>w>_2xWCwS*TL*e?pk<0 z!7;|%_k85-SiXb;4FaD6{kAdA-(r$LG}}_t%n0N$;LWyC*X;#( z>3xxC&(<)&_5$BCa~~fX>3(=l+aZ5d#}@Wxt}N_6 z;Cq7G6HU~l>C&Zv)Kp{PzYdAZP5N_lTv)uR#S})7{~8u47)(2 zte0@%8DB;xX1_o>{14ZBjd$kO@i)c0*zg4K61@JBcccF81lhJW%szp+NGQ7Y!!VIpjHe+t{kv=BoSGIUN1UQQ7pg%J&TXJnKUHosR5~RTjU^f`{#i4`U zzx@5{{*i;-d4BgRA7&zmzl!&K^9?-mqu24sk6(w^HnbV(N5HQMKdM^2^j}PmQD`s< z#?z2@5YB>p3(~XAnEMyIzrrU9_tRAtAG!MRYaD#brSIV_m%axlap^Dr0gM$GFEwmw z+=iA^B*%+%RX{KTHjJ!w9DqSq(5cY(P+##lW%#L9sIQ7#cNqqR`i+akiJL(nOxO@c zc>=%D34;NjuI%RnN67`J7(AD7WtaA7y`-Mi%4Q;rlG?04JJ*V?Lx!uRNw0O-k?8AY zC8W)UK3?{`>%GnbfAi7}%%zWsB>pOvUU)qKK{=1Ce23wEl5lDGWwfIGw4#{(Fl7co z{ovWaLa@ajTKJ034+X)&Sq0}0w!w!h9)dr)?!^=SuEz^+e)3;XeZit%;>Jtgy?`Ki z8gLC(4FZc3aJaHoe6X!mtL-OL@YD~{1|;i)RyF_=W;iott?J^6>hKm3YOOxvMQsyR zBVhfW2??T_1MtY$+6jd-qwK`tOsOr9U18WO&8)4a$45r^*7|=`ItmzdojR0R;(Ri- z*S97*UKKG_%VN#8&*0w>8}sF2v38%;GA&X8OGT#fP;J`8N_OpPXT%8FF?QL%SvWD(DnK38|;G1!L?pX_DvPus$GSK@?_B8_AN=d$)`0x+kaYEYY zUEh2?AGqpsIH{b^d7m&%}5mu%4(yo-_bj zA_g$>C$=R+0cm6xpk0A!yP&xQ5WPuNC2jNsKvDpR8v0^=XyN>T1QU?34ix$%rvg5C z_Q~Pn21aIoALTi{ix95>E`=r;dE&1bYrsZ^;QT>vwg$YpUc5Z<$w%6Yh<(7K+|q0a zaceQ_{}?EN_~`fF!->SZzxi4|{KC&baoBnS|GNWTei7k%2q%%MVn)A|R!pFJUhx5^ ze%{0E>B3Gk<6M>9=ZpFB{gmCo_4Ds6>=;-;5s$aM>f6jNj!c@j;4PPJz?blYFtDa= zG_P`|+}TEUm6<&Y_yY)*T8urWNBoV;4cuTnOq%Q&#@M!a!N|TxHubc21BP}w$_;V+ z2ByfUUpXeS9C&rEKy6Ae5Lk5u;2IOiAQsvflO84mE)eC}GvF!C)Yx`1~R zPG_#jh!zv)WA7O$Fh$?C8a=9brFcuv2VN}WVl#ac+5bFvaP=J>nc2y#hGjEa)MkJU3YX6Lz7Z3Mv>vjGX` zj}Kdg&O`To*p)43`b9;(7!wF{08nQFS&S7B`qMwLem!s9xN*`t>A%idQ)|FNuH2FF zy~||!0}JD_$5c~VF#wria&{oQtZm4f<;5mM#@gK)LT1y(%nHxF38$NS`#TK4pQKz0 zRYTm&V3@H@jHHHFn^9UhFxABS`u{aeCy{ctW?-(hgEQ#DY&Gyo zWEXT~hi{SD4^5eyR`u^49L{8U-wQrhYrOY<>$S(VJ>LHG4ODx}4`61oOkdW`L&4L^ zE_jw<=Yg#Zqe$!T){IxxOy}ul|3n+Lg89JhCbn7t&W+6gN{C_1f?8k?W6kty&ID>N zQ)dEMgH{+)lse{E(>f?Wm{v|8dokpWyH(gg>fII z&8G6O8T`L1KUKLls4!fy;oEOLs(1g>%l-+>Ve$!7#^i_Aec$0_OEZO54|v{M1;LpH z)tRH+$;owb;*PvubPQL-)fBCrum>b=7;U3b#7!pKEECsn`D_aY5IX+Kpdlqy+jvT# znj{e8W+_ad+TpH#>;Vfwi)XZVW&i2)lS_Yg-@_CAeU6F$IO`Mw;PBxBvuEVm2K&qB z8hR~o#gWR7@rhE@I?`8Wus=KRzJlxQFAqf=n8_>Md#Y{fLBLWegP- zG2h_%gs}s6fc+8|+zNUGf&)N5(62xvRIdB0=ly#~xIy*#0*s*6vOy3Krs}gwdsmrH znLs{mLkl(9wzmuD0nLyL<^+LGHMj`uB4ADZIb&Z;*!K|$AemCX))dL|WCLNLq!}H4 z>OWRActmhlc{qP)?xKF6RZ`H9^%j6WOMYQdPe9}`RDHmzXCOlKQAr@6#snnM0d~$% zapYDr?)&>wP6FUv4?TKZKGG=uZSBdmhi~^C){|>52X>kn?*hBbAUi(STVL$ zHrXQ>BZR814;9L$B}WzE=j#9uOfoN9Xxlr}2u{X0lhC4nju{Iu%c08W-e|LL=n$b6%~4 z2ay!*)*_OCRcq8ku4^TArr=cyd%#WwTZZv$V3=?Pq@BRF%HRD!Uu7$$bfrg_o_!wS z(p~dFbHV1Cf_a#A7<3pGZiW{j*;LUzZKMzP8zz=VuWV@ZR;86@*^c;Mob8EbacIE+ z3}8rNw}l3Ll39OZpKnApNHT>qCJ<2%gt1O=UMA2_0;dD6?AwQyEa4s7w;%i0*aRP2 zcS^0n{E~IJK+iCEo!}`z_fbbCG>T~l9Kp{I=9c|DzZAAk5MbW z8zpp7iJwLW5!TBQ1{3vGJIb)r!1^%;K*J~)#VAz1d8=pWF_1a1EJmk^4l_DKLbncT zwAA;l zsAubnMFNp(4~Txuj^<1ta+^+!OUeY#%8vTij&|ls@BGdmW2gSnghvSd+uF6{%X1lF zC&f~Md^SH{9+w;TDxqgsm<2gTiQ`rR(pm8J&M$m#ckblFIUDU2R?&r1$h>9@Ogup4 zSd|eIsRY(%dmgl6XUnwllL8?=apNcW{2M;Q@~(EaJz~2(FZ*@OdJ(TRYz^Upqm?&H zH~Q6EKVk@r5H*lG^2h81&C2RT*JfsIhRv(H)uIwcN83x^f5rf@kHWb%f(i43okFly!Ri8f) zZ>-o$pKaomrL4EQbGVe*aMizZ&4p*Q){Bk+A6a)YA6a)YzDf*aH-VAQWpb;)t}^&j z;JJq7nr8DE&ldl{e8V0y?CWp3f5)%0!AJ*{)M3>RAx#I2eh~E!F*@Rbh0Caw1D7FJ z9+`mFnRvYQlW*m%pL{EOx3%F>rf~b;?=>rZ0<*6H4;faDe#9|iZM6}^>m>*xCKV(` z)VS^(ozqg6Dr!9uyLY`mlJ}__E92LMX9d(PKygNA6?<^&tzle&eiL6b+0Fm& z{1*YVVd)W%u}?Jm;iX7|ks<+j<{yVPnt0^T`)e@k$NYaW3qECL4_BCEg@M-Ha`At4 zg2&>7Zx&SAD~L0L+9(?x9pfWq{hLgaAR_H;(XW9Ran`4J$G`1*r^T6kocW}T;8DgD zD`_TN#)&=77R-P_j4vy&Z_YpE<(6jKY;AK3jeesB4}kT@h&M9RFUCnD@7`2WXwC%Y zn(T$Xc=OW2(3-wXh7Yb?!G}*@&OfeQaYP^7hRXK&S3jQ*pSqGDFh)xOLZ<{OzCFO+ zAHY=x3(dl$rxUmhnDHw!{cp|{Hs^479cM|_lqi2#`U1bcl#?1+G<#J&vPNRt7 z9oM}yY?G0_$YlfTTyQXueF^blgF7)RHxPa5$R9f8;-f!dNu(Xkf=GMC+u3R#T2eeG zejH^85e8Aave|Xf=Z_75jpP~YTjDg*$h7JoR!ll2`Yx3l*1gIjz<-1N@EiAjY1in$ zR^;W8ViD-Xk0SWZ;lo_e+XJDUU;$wO!)%oJT}0Ux7Ykl3Xp1eD8A*Oo z&5)}|JJr|Pi%pR&Abqbb*S4+fmxJGI{^>ZLZu0Px2fxJ?r(O-_69k9JkF*b#b8MH< z2UfMFXF~f?_%S!Mdvz{KR1}X9Ok`}0iv1QR;`$&Ix2)H$uqa{I-ytedSiCg;ZA3>v zVr+rtB~e1JU!N{j#9F2CJ;LKb;OAhUK>nRynR`~*xeT7y&UYS+r&MdXIA|rta}>A@ zX9o;UPWY&(9*c^9KXShA5;UrxaJUe3)u2Ucu#n&+gI59b4iD>E=eQr(>F@K0y>7?* z*PK+D7ZzSz6=OC7?>}=bmv7q4k4`(eM=&!emFFt!oeXx7&=nRg{^S&Mpq@DU{nfU7w_<5oUdf4n=jONCwa-2_(YYsZxBr!v*1ll2bt! z0$qkmYExN;CzKQ9ka9p?p=+%j(U~okyC0b|Yl%tcgFQ!%w}4&;TpIdp>j_R%`eFQ2 z@*80WeFmQka(e_kZ92f3y2|r?_+PecnFrIoIB8YfXMo=(#G7Gu^@z zZtve&Jo377i;GTLdHRAm%E}%tB@0L2PV`qz+Uh2B-xK7em78-T#Kw8#hGpj_4*W~L zWt!V$&y#C%F3H~5oMY;~*zK?UIXcY~Ql9@&bmR1ec^lq0S{V7{t(krFH3gT-g^L?r zt2})9P~h9$d0j83g$Qpf`g8E26VD6md@ybRvAnyhdCs&Vd&qZj*a zO}{o^e=p(jRNWU<%;UMS>y*w#+({hsW}EA-H4TRi>oId_a@W}w=7!2v`F3Vjx>e7LmVqkZdLe7szIZt%w)w!AHWDP-o%!bQK7z3r8&wy!a-Vej5a{B;!@ z8pb5IJ=@FU@|$gU9|hIi`z?NZ<(!ok_nOZu=s5N4U0e zi)(x%@0!e_(X)SQ)u`y;yxT$RZ*}+EaZVR{oQsoy-o*GcQo?odbLG>^{dqURDs%Sw z&c$+*dmY=+OYW9@^z64uaix>(qHJCmzWnQD=M?R^E90u?#GdwU@^W#%U2$&7n`~FS z)F~fn{xskF&9JZyH(rjBVd9wdprOof?iOKZ?$3B}?|kLb_$E!zE42omJO~p+^xJXy==MDLgA{tDr-ic3>s6JLk7?6qOak!YWkg} zg%&RrjE~#M-aqn3?#}Q=Ac-)9C^8C!KK(GEd%_b8Tb;y^M>2?yEKWADN|Iw z^UJo4zbVM6pZ%$PSw$T2s{q}pN>P1Zcqxd0tlOpUfvf+tI z?UT~(9^clem0Ty2TT6TzzTyQI#JK*IeY1P!w)ZYi?>v8Of6IGcgW0=JQp6fW*@us{ zk9l|8`r*6Pty{{Nofvt*cKN+Zi(3OqI%vIqb&}`fweH>y*SkXRUlz5B9W&JK-T8~Z z$eJoV+IMtkvDB3_zht*)_UyTJjyomQs`}FMAlq$ zCNIU~&+Dx_C^sk=_qKXoXAjE(UX8l8h-%R6dasQZTD#-B)ZDTE{k-wsP34j4vu+H; z-1>Xodo`KoJuS_5`!))By7$`UjaOo4^nAPO@!I4>-`JgxE5~QG=v&b6MV|FTpTBIU zJ4Fp&V7PR2bDz+m-4zzBySH*i=KetjW2#y&u^F(ky;DMN(bmKlVPotb~EZy7JTp@p` z$(Y5ZiN8&EYCf{DUY=LtyxvD&kKOZ5E_3;r2F@*?cnAMA>V(zZYJXnrmgj@JR|)T| z-mAEQziD+V{?j)zVxLX6OT2VAvF#ewT@!U9Pap0y-fzEOzr~wIKQGFzcyTFDx1ej; zx~;sKmAsu-2ing%xA11qPEj|ewJVGAN!4w9$8e-v{Nf9LHNMireDZ9Ac*`dauU#kY zsNUUU^ZY?06P9Q+pKo@1z`BZrKZkoQKJKb{XwI5>#_u~n^Du71^U+nP-qLEshzFxW zU%Ri`e(T{TL2TlDorhymem!PVeQ0Y^mD1GXQ%A|D51gm|AUkT1*MkQe6N`>)nQJ@A zA-jqB<&!(E{M@lur~SWc*@c~&?6IrcL`-n_^O*ClM%R8j{>Xmd#J9K8hd$V6<#^Me z&Eri|kB!(UnBuy6cg&fXqY5=AuPnY;`P=K5Rx`%w1~+*o=wGn9^xTfbGv4|gjz*T% z%;ptWs&#V6)@^diG`;}wQBzSl7n9}U)@uA&I)a(8uVP= zQmb^ct-^Zc=f#eXAK%@XqhkN&^jN#6&ii{_QuaFZx|8jT8zwvNo_yM8%JQwo&qJT? zZ4tle`P{v&Z>rst@$6=0Vpln>^mv-@RNket#!dXIdNp$USs1EOZR#^Tz0-(QDNAxn zr)wAI9g4{tb!q)mvl@jHcGc&8G1+A4JjdESrLSfC$hDOZ+8wW2*J*D_Pu@r~!xl$M zIy9d)W8JUa^<>`X_iFa@Q2(V}XDJBUluu}8YEW`%%=^{TH>!l%B+Bn=eQn3kx7iE- zyxF5e+vm^oMr=#GS~Ed+_l5wwQ47N|AFS^y|FBWH;N)HR`N>^EWv1R}ILl{4kM*Z_ z9^P?z^!?ROTbaFku<^>tV-s5u+jZ|$w$^Wx6>zkUx~;pW45FOH4x_+Ug*NU77! zNx^$c4l4f9c5K+Pb<0~W;Z^-PtJ%6LY=!IW*C~b=mjEU|ywAZAr zM#MngzS-_e_ny9P`F`G=*BjT$H)?o!*Xy-^yt*N8zH959Ww+ukHYe?M9V(Pl4tI}j z=O=sj%$tP21zsCIk<=-7>eNp?uBij=QxkYV|Q|(QWmle5<4l=+grK@Eg%$A0$6uXgoWeEI2yqoKNYUo{sxJc`U5z3f7?)zBL{d#d?E zW#0sDD_9#iVqdlWjv0}oqU?-bpFaKTE<3x8i2*ws+a{N`dS!LJvDc$Js-frCy_mFL zEvDvGq#A##%P!-eJRa;Dk+t7Aa-emeBCR<;+sRg)2%h)+uJha0As5egI;NKC`F!qD z=Bd%~x9gntYP3AxaosO3EF4<&eNkpKN_nrb(Z$P?s!BAU%rS3hp|ElAly%WhrrJ86 znl^g)AIF1!cX>0->A)l9VdG~CZ zg*$WV=9oXV-sX_2hekKwedTRv@paEjLwCP%sSdZieE4a~bnRg?o3yf;G`xw=o;_|h z@mPPZ+Wmn1LA#4>$cUut+aI3WdSztqCsEfk_vAC7cWx|cJIcP*p*`tae|k35tk_oT zLf66Sy1$Gve0t%I%bQiUC(G6ZmsPo}tghL)a$e%rs;J6vpW$yNY&^K7)m7fQ>D|(G zOVo6qWwpB0dc-7)oj0dnc8Srx$Ln$Y=&fn7XFGRO4SId|xas1VHb+_?jK8+y)-%g& z|55XGl@48hozX2~(b~A)*Z1ssnsc(%0mp%t7q1)G+D~mv!;#~{=00xGB7L&?FR!b~NA%Pzdxnd;i6!e{;LN2QU;H@a?woie)N=M@17m=E9f@%e%{lqjfxF#Gr6zi zJoC>+oNXTGup^rMrkh-KS?09rjd@A2yajy>tb?!T=e;!Vs=i{=XyNFe(`WBVxNv57 zjaSoaESF3%_j@`Fv zeG|{$qYl1aru14N$YivCmE-EeB}>Z&sa$3@wBNGkdh>+p@_3}F&`Mr@|ld_%H zM!tXDa^m?_SAMc9s!=MMG_crxi+V?;y!J_Pr>_3FwFXTI1Gd)-_#eVv$!;ca+mhFko=p`sr zQ4Y9kd~4Bkt>hJ+JyV}Fu^ZB6nxCxvgY1?2h7MG`uyFML&9R-|K6>-`K!?v}4oVQ|(;@>a%Y0CpvfR zce`Yw6|V#Dy+cs5w=J(-Sg>rfNqWUF`$lDr$`-bNJUs4D%`rox-(I#*YUkX3?oSUx z-`uh+;g#P_^B^1BzVz-aKV9|LgQKU!TSx7Fo;0@G>yet>e_8D~wnVe*k#3$lRZJQ_ z+E}gr%=TV%QyZVBn@n@%dgdNFfA4ghOGW6EPQ$8;F2>&jdeObNUpIw zo7Vnm@2c!;{pQzn?qBY6Z{;+V@sHoRKiSpaH=&4Eemm;*n@*=)`N~IZ1;Z55#;+V# za69(A{hz&tGdfE}VW~p6%P%wb{A( zT_(NrnAa#Y`;60_BQZ^14PEeLqFu{|`{SnE9ebzowo^{iuO&Dagd9pey5y2&<0hBP ziVv=exT*c9*-fL_A&R~8GMdF4DryrR9X?*K%%k98iQL_`jW$O4=Xu=G{QYg0(Kkoi z7u;O&ENJ+Krc;)m$X}k98tv9=kKVf(>x5nAUDATKWabN$`E{!RBp3mLW#I5k*^Q4E< zT$?25e($Zm&`3GSIZiEKxY0}r*Zri-phOgN*y-@81v zyvMTHT4xnTz8Umvt*l2=eTP??j%pzz)D06;X06%RrMj}XAjs1=>-TXzo*A!`i#q6) z#tYovGO~%$@U>ybF8@(=XM4rk<=29$x~Lqgc(i71xaQcP zh)p-2q#5@Q*fV&L#|zJ2Z!8+rTD4WfiPsy3ZfrRB=8!=uCL@APa+Za@j7l&np8n$G zy+yrO4^Qo*u6rU^;ACUjdFMp^YX|)16^(Q__RES87w6aMYu3!Vq8cWDwnvs=W{ZaX z8|Ebc)_rNe6P+9%Z%sbDIr+@N8Gb5fC+(ZGuf@#AeRrtaW_E2KQ@TCl{f-;Uoib8? zD=0Rpx_>87bI4nZktG)M>^xs}JH0gRS2tl|*uM5h8-{l6nVQ#S;JuqK?@st*l6m;> z2i}W^>NqmH{hnUlb#8F_w!lLVFUY=)-s^w1(UtID7kH09rarOZ+2uMt_V_J7T-5gP7p=4o-QFqD!EsNcIcqPs+R{Gdk1HuUGFq)WEVm z`@&w$yc<=JP4_&^c38K6{p*dF1&Oa5`>#zIw|C_o*;~)2`Hj&ExEl9Fy~iFm&#C;4 zj}M2=-Dh$)^^8}UaayooZi-SV(xaF;z58v`?39nWsFziKz{Xjbz3G<~PZ zWv5R3ez?u$@Y_!ZK3#n}vij;C%b2GbHM6!ib~qj|Z?;bI0-HjeWYa0D|LAXVvT9D3 z)4n!JvjST$Tq5f}E-$+Ic?WrIO{44mb{%%iz0tuo{oFpIafdf$kLWBsF=>_jks-R% z9x3ZqY>)Nd^l0#|z*A)%;&&WrWwG<+o*PBoez)t?A$*qZ`^;+%%6k%=wtxk=Ry^Sc^Z!#a2gTXgO>(IwkK-U@vqyKQrnpHX(op+w35$;r*LeO?Gg-XJf0vO9PimTW#U zCv9hB@P?b2#shx?&j#Z9cO`NX%uI*4z>~rR5 zpN5^QUwf?^V$!!?+T$L}&3h&-zEQc(gJ&^r))O7r#L%LTF9KA8OWJ8%r%*C_3bp4qjQ-#GVU+I=_3o?lk9BcphdRD%y<^3~iGOh{y5-k@q$43L7jXuz`M_7_zz);@qzQJXW z1(|{=f27|lUo>WirD+JI_`bg&_vjAYYOj3#+twrNB!SEJ7y}g(DK1yJ<{3p zPP041PqdqLX<*xp(-s|h5H{NEr}^HE+sUbB)AdI*-aF(}*QJlQcfFu4o4mE1`JB{dOIM8z^PS(; zZPeD$f+3L$qt55{_dmDq%)$;$PIqsj6#n5qYs0D%&zT-IJJUJ+F2Td8*8s;gJv^sq z;dv5$Lx%pZN}u5&Lk_eiEV$SI{rA6D;J;VkzgOV@{uP*P<3%K}9!DZsc#{|#UlM0G zp2XSGtsjZ&=11Zw=yTw|kN?ao;O_2Dc92-i(;7=6tUZCTBv5}em4Wt?t-S=1HsdU! zY`ptN+4zi%vh^MpZR@ZW^Wz<(eAc~>CZ(hIzUb8Mq*#*uJq z&vxLyZ-lj1+GOi-KZE-}BW=82f&2H-wmwxcw!SQ4?ffcX3##L~jX#5}CbbLyef;NK zfiUYaB+zgo2?O_`(D^XyvGU>8WBtJWUU2^y+%w>v0q+cWXTUoXW9P?!cLrQD@!iHV z3Ed`8TuJEWZwmb9oc#auBMldhB0)i6B+POw39%g0EX>k#Xt)*2`+JV}ap2us!aL=h zIvz$+4674DcnIwDvJ4p`y_Q3!1tNNdD`k`iHNtnqP5@zp3bzddi%5y5Xe-52z zDEE|is`ISAL+5Mto$5NrJ2+;V z@+V!ogtPdk4=vDcESYHGMZ)c1@2$offcI_T)?U?=d-(YebY9xdQ+;Q72j?vB-2<7F z?vt3*?vpA~y9c=g|McPi$L(KU&5y{b`w@B7i9|ugpD3zMB2D-Mh`e%ujJ#q{BPHdD zO%;^_n#wDNHc(Rwkl~rnB`UmVqRfk8p~8>;q31lw)|14TQ|>)Y!2MCN&VQ8qI<~%! zzKeJdV$ym9G3h;ms{v1dq`AfX(Bu72+?l+*FVS#|CY`$Y5(U)>M5WDS@IRrGg6c#I z1=WBF3aWucifWVADyjwTQdA4xtEd*TS4ll=8?aPKJv>X97covneXL!1{P8J1^FE zG4J3$gvsa`!esUg4an#fLUOx>ef6<@>mQ788%8``MvzG49T7GGWRgV$39$u z?d4U*PXzy`q4!nLdqz<;P{ez%i2DzCXO#I-Kr{p1fmlXG5J&MsMG${fRggFt(Bi31 zkyY22LU@3RMk>+JNc!wyCt+kG%-V|tJNU?e`%rNI7Bw9%w`2AGW88~ly8!4s%l(_Q z?m<^!11dcOuX-#)E-Nb}^Lp#=Ps%wEA{5z!_|NlEp8>YHCr?;lTQxWtp`{S(Nve(@yN zIl4h%-x%G3K2a0$dq@5Z8*mdg<~3x1#u-dTk6`$rU^W(^GD_u@<|ecr6@&AkaSWAj zdaj6fm{7|x6+srGZU&l<+#Fywe9}bIpM(>PJQ|HR@gROC?qsZy8}T+C{LRO&B8M1X z1yJ^buOGj?mV0XNson?I@$(`d|AC!X71ZZl#JOe)1HOS&5z-iSO+YLCIZz52;0d(S ziK$D_3z*i@R&zXA7IQ@RIs#Fq89MYiKf zYL8IqUH_2ha|R@lIRg^OEaxOLwSR1*=}vJLB~Gyg#ZIvoi~2^_6!wW?AOj3!fPs%e zoDs%kIZ_!2`9KC}ZGz^jv~EGqZ?cpOzy>go0VdFVM2)}MPnRc}4x8a;;$i1&;@-s9 z$eoN6x)N`_0bhMia;nJfRN(Jb#>T*wGJSpzk_%5=p&a?jgbG%c|(^!CK$~llB zg-krpJglaj)uXdXBvGe6K`U2s9^r_Cf=xVSzf2d{uO8QYvVjgd@tr6@m+0f z_YvMTVdEvb&gyu5t{HXhEEYWNY(@%~@O-ekMvg4ypQp{qK)4O!ew(pv5bqtR&GE%P zpR47H>pG%Z-Y&$}@|5?~9+SX-z?WW%FaBPVLlT+plt#*3r;s@=sh#GyObMIilKjUk z=VS(Qz(5Wd$N?jk15O5FK9B+G3lNX8ISGwX>G6yG0F`ad4|t3Gz*q)#Zm|tu{2>Dq zOouUkCPNt?WB1G6MuVfr84c<@)&Ty%U;r5_bo<-$QCAOObze?pyb8EK`f={5ucvVy z^j;j-*U@YdWWv|UCw$#PjN^Jyn8Fs`m9FPw ztX`7mQU598{t2$5>lE;w+U=jw`-QIQqF*`+gtx9ezEthVstocRj9k@)#Zfjz5O0c;A_?TOvdG zhyA1QL`+;EuIYbZ_rX8CcHn*k^u9IK{S@T#Kk~8AEt4#81qNo2c>^lc^`?lXG+SnlU}CXsog6Ujoi zEXY8H+=79b1waMBaL8ch56s}?z*Qs%R0ddE!1;q(84$+e1u9L6N*C|K#owwBP*Nye0Z9a|j z>e&0Y;5$P_BeNxBfT(Gwp{AcE5iu?OOrpvwWMeq!J|PVW0mlZwT!C&$g+Q-_;V>23 zPof&1m9*EH-7MP1yGWexeM0wf-Gc7JhN`Ca_;F)+E8O$R!om4u+2Aay`@+S8vUdW^ zl0i8vmJZHkDCq0O0F?($4mev-2A_b~fRzCn7t(lu#)LF45XSc;_t#WsL}{r_{M{s7C-1}?%@B0-`D4!&F9vr2{PKj{u6Z_I+pY4nwdbanCR$DBRt(AqM=*T5}XDI^rqbb7!4NF zv8|73kP8+s)O2UMgZqryI-hjDC((UsJMW{mssb6PbDrP&;*!c4vdXmx{yr0WpG~KSiv_7~ z`!EjRYC?5wz)w=Sz!1g@ae$|R8~Ar+Mhaav{G`{vBS1#z)ye?uJyB4b)EeBcGm zEnhm$|2X%k+dM#w=cuNYN>tQKKg+%N+Ux30Bc?htVFOBtKzmvP@I6FBf5ue}{h7?C zF$15?VEB47kN8-I%!sx1eaFT8;9oS3PyHm#o2bnM?@u#&2AljS{^z-;kcETN$)Z8o zTHt_$D(cho{4S0INztSGTXnw zjp4sVO>G<zkVRqln%YWE@u=xZty%ZG?$ zfck=gnN{;$(@!HeD4y$*G6XdO^_k9zt*7-*Xo~yBG%Rw8Yf#c3wPMHc2Dy$Ajlq9& z)CAh5cMsOad5wa6&%(K0KyADF!+L;JF8F~DfDPc{K*~QeQs3psaG{Ib2z_b(L%@HC zilSQRsoHwJsJ4Urp3U(-;GWItl2tTnef_7n{?z*#!cwAXIGYHa7Z6Q@*`qaua~=as zUChR7F=jD(!Wl39tis-YEbilUpY7p53>C68yH{AVtX^Rx-Raw6!193wWPw`_f&R9I zKc4|zuA+9G+IXt>67Gi8kwGiyCcSggToP6qO+FL(=?IfQ-q!4L2kxuna2f3`*p z{l^$V0-4%3mK`H1>>EY$G3E^YC*U8sO9-Jd4E{-?V{k+G03G;%Fv$K{)I4jrT<}YL z0P+As_y7a=0DYI&;C=|U5?Y}1L{UAI51)TQ?DNHSy}I0|h-&q09j5~I*%8R;iHQld z{onS`G@45^g>#8Oe_mrPVOg}6LD_4(2Iuv`$%O{wqnAn6yBIt4?x8*^$@!4;)4B<4 zzIZJEZl+^c_w1e#B-QEbdpYO3W)K?h&l^}M3+^JJ`|m!`eb{`e_iIGlulJbBP*7P| z4HtMHz&io>O1$uk?061a+Y|Zx+WelY+q3n0TFc3K zfqcF{w&LA<^Hx*83^Gtov<&9A)-otx05Dnx^Q2IQ=M&xvumef%Nndn~dr;G*{4XAyOF45v9Dj$#^OS!Y+fm;x_W6`& z%Ktj(z7+p&VFQ+}axXBR8I&Zmd{7Q@##r)uLT9q#3y-1xfhIZllGHvwkuC2(qjTr^+2u`GzfVh8v{`O4-VJsCnx?5@DBdNdEovW9m8dH zUt;&^xE&p@qxE|_R>N{n>$J~#+WCEX+Q{jCz(dDyF3~nzK(q|!>uVV|l9`jW>S z=qYM?(Ekh#@c#?V<37nhtN(TE|6}NX!kQsP?N+)MlI25lfgf{?#K(>th)^UrjF*kH z@e0Bmhu3s$n5zem!k7^9KyFNk@x_<`>cNs20C^zzceylN-`PUEKPBE|^y5udLS6qj zTh|v*-AAtm_4(`=4&ptQ`%J3$lzUpMeaY7;9E@6ycsJj*rG7xiU?J($GYfLCa5V4| zU?3ABKo$%rs2r3thK93h0&FK&C$r;xT<xRzw z>~DR~hv*HUwNUhkd9{ZAFQoiSjSIsE&^*wC9TW04rgtGwbC|P zyb@sW8vIk#$^vT_49aUepuGccZ){uB6rCgERX|GAvMZr&$~|F6L2yR9CUFN-a>pZU$UzRm%C=0%$0 z81zB0VG5ccRRXYyrE!ScNYY@3UigoZUx` z$Lc;ePE)|uY4u79^fU`e7t?Q#|HQ{2-fEgHBzlHRiH`A-?m9+G?f{GqD+iDTDLH_T zU`&l?JrA=Be=e2xL;oedAG#mT_F2%n8u~8IA-{H-58IAhn-ukl7XPGVJJ)#%L9Ld| zbxl>9=bCn$j^)t29pgQ04o`D=F8;5z0c;NFUT_1x-YuSKK`8fYJc=ay{3o9zIe20` z40D^U$F&0gd$_)UQRoX8Ar%AG84sehV44R$fe&yMADg0T1kp2}N}&7MY>f}PFUj-S z9u1MtXUA>mSPkm+Y~6O+CQbdBZBVxn@9Mj^(0!t7w1nsxEpDu1yllFT(NgeFLFHfx z?19)OEJSRu@ZAv0tbO1g_5Mk$?WesobS$0LP{sV`gZ~2b-Jr&H1>Ad<^owbVx+}?d zi6Zk5>(NJV3!q9|2?F=Nq5~ zL{_^Ov-|s3{e!VU5^c%$!j{zU3!!x&IwmYN9w_GjCi1`T;$u`aN=HwAJP_xLeD5vW zpC{4%dUl^3tD!Y})NHxBoe||Kd;A|5OI*$^n%DEyG2(dYDdiO6xIc zPI`}^s?XSe@E^tYV4?q_5_R_-sJ-@{)-SQ~Oy_tq6Slw9Ik_3eY0FWcVaM;`12`Fg z4?rHk#euK^wK9;idfDKd&dc4h$)+ADKjgf>b|(?mzBC`Cz3};=R_F_(Jt4F&OxzQK z7*NbVtpm~=5d5znsn@@$^lvG%`k&Aay8jE;vqk;BSodjv2KD)z-4}6>8V%~T!a3VC z4Cl252;X9n^e(^axsgd3(J@&=bWK+E1s>^|tYm8CppGr5)-+ramD|I=Gx%Qu8&F-} z{VdaLP?Myi)|jpalt+Mwj{cb-W-YUvvwbK`ck$oi@MBH)M{L{^~#7!w3O{t0JLi=B)ynRT>RhxTqsY(PC3Kpmgjfj5W;0@6oC z(D6XBY*4oNu>YmoczZhck;eaCy}X)eh(EMM*Kkd7P|7ZE9w8{6|EUeo0j@&^z`sZaB>sT)1vSVKf>jMSl2nYJWOc_aHypzz zfd8vp4nXriJU9k*M8$kU4jRnOrP(CpIDRM&Rd9GRL z_5au8dG)U!#y?3k)~<@S^lgY<4Y z1gXSSSVVib!eIL=#kC&J?~8Sxv-_OxmxFuM=TM`;_5T&|nTDRpa$;`!-Rv<;RuZYS76vM{$J%QjXv%Q3tg_}_^dP<3q%@L3zcl%g+# zZz*E_CF=X=)yq(b<2rbva2CVmo47{%^15$p#c-bG?hD-U#8AwL=D9Atr#(Y9B z*D^v0{AYsy`!ojlxDAM7eE{Nsx;8*01Dr2lbAk%k0$p@Rz48(elYi^)Fp}d^z2Dy zJ7au*NH&?>|J!9EqkAZsY3)bS(7T>tAH>cNPqPhZkYMj`fbris;Qu9^_w-5r#eE=L z{`VYX0A3@FJY>9#hJ2O>U86M2x5y=Y?OZkFe7|tDzPh%b_4{>opUusvZw5Dy!P9ts z^Pd|USZpQ+*4xn^K;_`uAKKRIiLS+lX8L9uc4FW74Jrdtu|TaKNImMZlIWv$B!1*! zu60hI2!b&nQrJ7H3-}L#4fstQ2hd#bOJzV37t*-ECFL#nKM#4B3ycTMX1gS}oI4;* zwiNY*S^bk}K1twD$jp98WWiuh^pAah{E^NT&vc9e(Eow)%Cw&7{XqYEnnO@y^feoy z*C7S-0x!|t7x2%{cb1+HKJ8i?r->4)65~-{A|+6 zVJYOGa4hOQuV`N`?Z0Dtv)H(Ap0wX*>oPPhr@onTqi?qHkmap2H82n6h%{Rz^|1{n)DDPk0VfAEN8s`V*akZ1=LTLM zz-LouJ5L!m%Xx}5zRw5$9ZUNqH=5=&g`nq~Am<<@0D3-2kyA8*Eh4Z>?Aicyy@_1# zp6M9T9Q-R~^awJ=n5H+zwpW1vYv8|{?E$4dfTDS!bUqLr|F1v(N9P5K>c7i;jfZyi zGaiDuV3HxfI@hQE3r)xXVm_d-73%zpxmqu^{j*^ArEULWaKBVq_rd)J;Aci?zWJqr z#nu8t%N_DYR(pxinnV5H`jXcM<{RJxekMY*jV{lTWXCuV~q#K2c_nfkMc@O>sW(aT!2fAd&;r2qm&0mWhvSYfwvgj=%X0ImK4w z7G;p9RLE5@_#X@Y-N1jpVyEbS;J+XE9|-?62K{E68DwpqBLjJq^`#e2be0 z;?K@=qrH!;?f>+=U~c@!Y}h*Jzp9_LLgE+=JSIk{;ot0AoNUy(c#=?i+0i z87PDwxCnl#rRxHmE#PuPnjebo0ksLzb^&~|ui5t@EA)Hx`_zumebD(Hur0LiT!G(t zhTnTq*f;tKWbq03e+K?5z<(vJS!B_50Yr0v=(rZ0(~^X_?A(0N+Ic_{-!C-}Kr|n$ z8goLIV@{y@c$1&7X2_TGF1~KyopiC8O$^ma2yFjQ^x;0FeVMhspXPiRlVjsP*naq9 z#C&4iXJGfi{|?5`au;K0wfBLM)&5c=s{I0Lkwg+W=c#*nw#9 za|%4Yp4JaFV%8tj`UG*EfRhDjx!|7D&pFiLovk&{ID>-LA7~wl<|5!;YCQmM?g#B( zqGMUKpNaN0QQfC~j^a5$obRV|LAm+BlH8B_ev6T>Q2Q@I50D&sK(Iy()&{{kG2irX zUqN^VrGzJ(*$BP5q2T|OsNN^e`9-#$^1oKf_HUuO&p`LV{~pH3Y9C`{eE>Lg$Jpk` zTyQ@C=xl6%oEY0sSvWw9ttj>pbK70W_jY}7xO!&mh_>Yt!~h#h08tD88$e?Ywk9OW z1!pujN=Cf!_c9>oj_pGC(W_tDLQ6O|8T`MNs`-4x_fy-?@z3hM+DeX~ul13ymy z|L>$@0FHlJ7iPx^cjy@{X-a!YzUEQXD~`-?pG>GNfR3_p8swlQ#)z%JUlecvcqEpE z&u`=E@J?e38fQ|S{}lI<^@DFTdfta!<_)i7@uc)sP z=Yd>JP{RLd_B_MqetHmJ=zBIh^A3_bHsi@7<0R)WAq9B+1O9w`|I(4j4`n}Ce1%SKgik$ zL-+~s54qX>#=vsdBk+F*e&jaf=?>yaVDkg;{}TMy)bcNt2Xgv<4LN|hu2c=+t4>zx z%lYnD19I3T)b?nf2$#Dpa*d#`J7WCM0-OgSmR|v`e+M4n{Tj-*7*yXmo#%L`JxJ32 zp4xnldusRL^LuhWpW1zK%+LA#THWVrK0Z%z4~sp>`+PC(qY!N2PSF2NLX5@`x*qaB z^f7bpV1n1QLOTqdu>lfz=7xknD9f-^=ku>8ZGaQttQ@c$YA zUxVZFqWl{Q=YQ-pzWcr6{;1oyP9}3uuPJiQBsu*fNt#Oti5TKb_L-J8p580I>!iV6 zHaduTDI)w2Y;=zm`IFbo+9$OXZ$wk~e+zd1GGe|{SOaPg>V4&L-6jOX@7rVEt5SrOZzIfs$9kaSu&zlEdoHvO^k4a) z^))Ti#o&KA(KTI8$9ufM|0(EXwW#(ZmH+K#dF5h2nghVrOF&>YO7~g*kwYMl zq_qUfKaDL|-Ivq=x%i*jA8~I8%}2rgWIfB3MBCu&V!*GthPp|*#5zZpC`ITI#Ry%Z z9H!e?F-%)oAxzstK2$qfK2&R?LWtHm#bE8Hib2}dij%Y%g+MJ10a}crzZRo1Ugy2G zx6wKZoE>A&#h4d{|Vsd4{E1H^&e3VD3$@_fvhj6<(<=eiQoTF z_gVhMwqKfmas5X;4j|6|*nWUqE#cyZ7!UlR=Pe(rM>GO-h*A{zj?^PvLn-%qa!TR4 zLx5RI;W~dPh3ULf3e{m0DMEl?Z4N=$QW=<3D+3DtnvDEJ4Mq;=Izdp;b-duWF5~%e zZ6>HH4<9gq^c>oQ2)r!G_`aVWulu1V_zgSLL4O8?F2?CqEw;;`Mq_|HEeZO14mF^6 ztPQ6%Aoz4z3t(+L8|R5Qr}3Uh??15nQo2uTiqy7nzMqZ(i20YU|4DiQ%5@BvHV3}p zj6VJ+U89Xi$5=C#cZEou2H;V!A>1!F!ZmBKZFUs8pV;$3PAk$N|I3Kz%u=ZwtOm1{m;v7YLB^(QJ$@ zk@toSBz^bk{G&VIW4u@1YO83hW-~E1+lE@+)^>>DtRVxzkbw=rRmAo$*!Z4}^Te^9 zRJ`|5?!~$<&ikeEzPi0&v@X31`APfl;a>%|p4xe(7=6Oe5YnxRa>%c;153L3#bjC@Pi%jgz2yU*%=4VUwM z)b@k_#lK)~KqugvPDwddNGMdI^@wt`ZYz}-{gKKs`bU5&j*8kSwU6?BG(@e?jsRHrox95!skzg&qvLFLkV)e0{Hz@ zGwGGN--OO2>`n|#eF{ab$5&NCRhz|pS@!&sPDNMH*GO!ym@IX2*U~2=cAJDAz19fe{Yj8gX@gR}&;{W}5 z{U7}XbRPV#G)2$J4Q{-zb}kslJ-b%GWpJ!M2CcxoANanck^tTT zRTc)=N`w21{f@MJ#0l@K#ObetkMM&nuuzE+w(Hs}pC}`DP@Jqw6vGiqhM{f{p@UkH zc3a3mFYvz@^?_HY4{@~taV+pL8{mytU>tv+ycfSSKom!7{2yib^QQ&Q319_C%?lQ- z5m386z#mv60{Z{y`mb_=0Z~p6vfL{t=&OV4QgB)!;W`nV{~aI~_?tfKcfoowl4~#h zLCz!4cvaCO1 z{YvEbG}mK$kzniDo@CM5Fo!jT^E%S;O6iAEk`d92Hy~;WhKTP?8-c&kz-5+qX}*)e zKMO-_Kfn`wHcKr@Xr!8C)Bs!Xts2`Ku-A69IG{cAh9F z!q+DlM1%K7b+{LEJ_VcuY9Hbi-V0TkV%U`9-H@~kav%!P=91s}swZgAQV2$T5Cp#v ztS!Lyl(Y@_Xf8O8{}#IMF6aIKkAGd`#i%DNl!eYGNyU2HT!dQgX^-ksZG*Y!RVriY zP)>rcPliuUM4Xps& zQl)Yb1yo9Mf%mUnOq;iZ5ecDBCNBwLIy1jx2uE@6vjXw(;U7 z*e>DheI1(*-szf=>^hP8n5)uA3q5Oy>)9AjjmCP3LKX175U8x>UP{-+oJ;r?@t%gQ z7^z0j@!Uft(VPrG6EQF6^S=H|&K){W^`F8B{NIq&1K9CF)P%%7fb!4f0ZTL$2vHs+ z|0TQsmwiW5A3cH;!r2&mpVwGjs}OT{!|?mKP(R?IZLk>oUEC4epJ97Ysm|Bcd(2Uo zOEFpBs1$P-%7`jzdCCcTkO3hNysrUjV>{~Gsg1ACyO?XbP5&qksAU)_;WMI=WcdTm zj1c({T|lQf_&*@cKU)(P^H1x+9RJ(8cxg20;-&dRe*1fOqNzWFXaQ`WsQ%nW=oNK_ z{$mWjTmkcih@M6E_x=R>59hP%gMxQ^;9-5<#cRXjx8FhjhHF6vRl>W$-y6`lTpNCW zYklq|wmy}$>$N%#&ebI_X4Et4A>F7_J;NwcE!Rj!J=5sB&yDUtF&MS|5H=26ARPlp z#sp9cV#fnG{`YhpCujrwkl+5Eov7&*5%aOty}%5^tb^9DOsJ*Nb&u zTJOcSUZnFZ@6vo{g8Ppl!}w3gg&yR9sAm{{@97~9)Fuic+700YX21s2sRfAhK(-FR z@xQn0SbiJeyMOmvcfjWt6BbziRYk9;0pKSI=dMVkzqqP8~5FMHa&i@qu?3ggi zzhFlfPt9h)cm3wK{;?MB&#Ko2dT*3P=Ljocj_?%B6}%(4hl@VFmj_)Zgsu-Zdjr(IyXf8kxOW8oO~~JJP4Yq(eqRyXud2&E ze7`uSqduSNKIh|M&yAPxGEJ1B>lVNzj&l=6J(~j$TMF@O^(;KkGWiSNHCE3uB5G;h zm;;JPfbFOCpysFt?x*~-KHy_@AddePon7ommyzFZcfLgiDF5m@(0}bbW2_Oe6Kh0N zbL#?%)(E2a@WFg>I%gD|R6!2Xv~*`T7GRA?0q*(nH4naU7WhX_VNli;yzi;2@8G_c zclvu((Eaf$$^7=v{rTX(rY`^BTpaJ!a{mT;U&u>0>7)uQQx0?)H#qZ zd_noA_wvFT5xucK)MIu{VD^4)^xkb$2IzgfuwG2996SX#{+haT8)^ur6M4)j(=;mi z?62wT!`GWkh2EDEUEwSmQzpTd(AZ97=Oy|sk~7Gg-i&(!yJS0+c->s^UkU!Xu^TDB zFX5l__Z8~t#%a*~*5IG0q#G0UY*V6^W7-9r?_zn+1^>CGA3`h>*(QHNPBa1XtuoMQ zycuFuErNbQAMpR?Z}@))85jk8_WJ#kAF}*o-G8k8k9+^*uIJVXqWACQ?&()+3qHjc4t=6z|+U!vy*z8&pE9 zQZWwq3jB-4@I?6@YCIC|Io*E^+n)m6Z%OkwzVey%CO)&so1;N^jPG4iFJ z)Ay)c zwoXRAmx;Baa$K=i(DPau;O^zg%7BzD5ZQyr8v4_xKn@hub*ExZDP3cL-Un(r_60yL zBtSliwWZqwVc`58=S#%*2;=St#@ZLi7Tq^&PKEj)uTALNy9@rgo{f5OpA`SEzL^0Mxbv z;(G&%?+GZnU!?9-hG!T1OgYT-0qx0Rb)SyS*XN&$`QC%`ELE!aX|VS|{Rem@{CqP4 zo;~>mW^Vw7MWJXbddZ&>*ZG_?D9a?jA99 z3br3Ad-Qz(0?Uf?x$s{ zcJJIX^oC(RiEeZB2hL^Z4p9B4{{Q3oUp4%Hf@L6&3@T8szpMYX*F5l#^+FK~rnkX% zF65wwTQ{2CE9PTzAbqdzzrF7>x88i6dqQ*fjHWh)#+&MnlWLShjcV9leDr2Z+kZ*T zj~m}Rrj}x)t(I&=_{pf*Cw=~sspnf0aE>uPb35R+0NhK1l?7=T$TNKgzv0IEj^Z!N z1yv~jQM&EX8#oL6KZF1O9RH^=FMu!aCm{U5|Gm23$F5o{xm>IrPVe`frUDt*1c=s- zr}qu1F9$R}pt%ACWPt^HpLceRaV~E7#A|9p#DAk*$neY(8D(Gl8t4!=7vlr|MRmVq zgRAJ{=*15e5Ec0QFB!Mx7eh{_T2QnRl$h@U7{OE#=yS0xsA~f<)DjJ3Q9Jn9W%}Sf zjYK`t-d}(oL7mo^JJ?qg|36@3KRQpaw*FIrF+kt8j-6%x9s52U9Fj=8_63Bmk&gAl z(=~B#*lnB)aB@)B9*As$b{<3hL#>Yx%f#om+~4FgJiVN175Bk!pf@$N&z_xsBh9}! z|4TG@qn2b4KpICA)fD*r)GxQs8u>xjSO?j0tQ1NPp5)E;)E@gATb z?%xZhgLeul2O=4e!~q=t*O3Qk0AI3u$#;~|2c{f_{z1$mmqTBuaD&&3tbKgvuAQm*Du6e)T$(x&b z^X|F#oO|xM=bU?PbhKN=MlWJbxEvl>)|`CukNsl*$CyVB zEJYtc7Q`)X0{wdeHvs?Pf5)^;_!>XRJn=R7jQ{-;nXwd^kzy%Lyc8elgR)MlMSq_; zK(7JHlOMD#NDT%g_U8xp6SP;%_06IGeSnYr_vnD4|CzF1`-#x~HmS~GKe)dCwI4cS zh5kzvV;@`b(#FgSF3y*l03Tch90Z<$7hVSr^URy*3$FlA0SADMz!2zfb-0!_u;mS9 zPlYQV{pb^TK-)@W&7@_3m;N#b%uoK9y_=inkL1&nM3e3V_&e0ZllnJnxGf$)%FpgzsLYD{bdf2Tnw`3a$eFM)_KVn zqYJ30@>90`AJV7JE%<`AWHw^Gzg@5SW!)$CA79#w4H$o*jty9q@vv=mMx)5p-+!I) zL-6`F|3+kiBLU#N5$F%-bEiCakrx6hls$eGuu_E-C=L8x&u##D+*r(8`R*oy#+tIlslW z{A;G&&w+b(X^?8Vx}#z}G)u3E@r%`x0ls?R?4(Njtfc0$mI|NwwNaZJfQPrh+otz18VC5)vN{P zS!YSUp1O&Ra<0b@us*X1Hl4iyKL1nkKw92Q9GHyz6&c?6;inzT(jtNXNm`uiMkNl$ zCc?TGNF@&9b;+4lBLjp7yn2AhfEfwzT4y9>Vi%}0G$pPF2m&6`GG4~Ij{Wn*1rsZ* z_XtQlk;IiOO?%q5BsCJKGaP=?FIs9-VMVbD`}Z*5Gl_?<;Q?&qr_BTRT4%VL0d>}8 z#DOZtw*z=Zk8^X5H}?d{-VjgRxZXoTY$3kJzhY|)6MJL*_2YXWvLH@fJlCP{eYy2M zDDq=z9uVJ%=mFk&Ky<;fgvDw>Zxu-_ZJk`V3??=R`me~y1teBb?+d9l-pw(ut_6? zPl;6V^IgPtrztPP7_9eE*!B+A{)?ang1EJ6#u)zcW zga^$1FCqsd9@3)=czEDB)`g+sH~le}_=v4@$Q6Ol&|aR>k#YTw9^1b8-sH14=bK4) zu{n$F`CZ$b;@8($=MCiKdAutRMXd+&(rsQisrs~zn6y0&Gxgv zL-kkP^gzgHrvifDfkW_saYi06*9FI!55|R0j8nELXYccM%yFnTb0av5yhl93WdOggAOCejeFg0u0!>0kJ8p&tjM_ZFeBjjyO&-`~ zDUN9zfxYl-uE5!GWT@FK`|sI@>9Rn}dunLwrSa)^V)eD}S@Qc!%zjzIA?%LC@%}jd z<81K$Ej$}KI*OdZ8pQxjCH5edqUf7wzN z7sUR%>wy!p4^PJz`P}v1yi@ds<{tmUG}o8d9mZjcB$l^s)}P-C`QN}p7p7k*@}FD> zj$+^|%>$#616mfK51<(73J+#~Z8z9O#Y~n>(m#UEAQ34%9MmXMq z2aMW0;Hwkp`QR{Xgua$Sd`9rVMWe%1NMY0|jM;N3Cj*BTNqlXb@|)eukJu84E4ddq zDY>__PeJ?&X8@g>%X@~C_@PVVj{y_oi$mZeWC-%4{;Jy^2peiwp+oIK@W9_gMnoB2 z9zZVCo)2ndLG<(RK<~(+n11+t7Oh&;j~ep#O#B%{?Fb2V*-*XPr>Ydf`#ljmO2Gq(OWN-vMH4s(HRR?mgC< zJ;T_;6*0cv-}6(@Ufvln$)ZB?qg3!vdoSRq@Idg_cmUn-%<_QqbD|$Z9*{R7<__Qy z{0Oh3E6RRa(~cM9n=JO}{&6==@@d05)jD6zu>#v`t2QC;0?epzGRA}%J zdnG(zAPbCdlm%zt1JiEgqbGXSjp9?%AiVHB!E-Ltq44o>5#;0}KBb=aKMm#0x5`>B zZ~%6|(V;46u)Q=W-~I)B01p6OS@3Q0z}IfzwIS*?gYbigC%krL;fwEr!kFiIrj3RC ztD)p#sK0*t9taxfP(cIXgMqe)pux5$07LVD<^xj}=sDqZKEP&BqaS$rKi%hr`Ck3<@r^YrV3b{y#nKy)^_4Vh$K&Q9*-jqkzwZ2ap8@bA!nX zUy}!?>4{!m@Ximy54Ctg_~KjPJ@{sPvw|f5umaYXtSRcRbJqimePlqSK+~XswmpCW z9~fuk1CM@iN+$U5f_L2@{NRPrA}35)A@br3a2Q@07c|7>A3TQjTF#rEj@JL5~ z^Px`vpkcA-APGNP?#-W<0>6OT-3nbWQZ-{92<&I;9oRqe4ax?*aEcGSGT}_T;Inqn zx}y(Y2yb{n+ZcvNorFhz4|E8gU_<}3iyitG&J{2*%yXzvTdqTYi))`kpoSS`byDrmSvg#z{0FQW%E{rg6$ zOM6-U1M{qX0{dE@h7Sx4!VAa)Ef>7=gI8DZkqstam^x&Q9+_|doOQ*6K+nLzQ2{~2 zqEz6BDCqCwL-=KM!EYXm=6R8-c^~G2e#`}VmL&M#TKM3<0DJ)b1$^a#caHGo1>px1 zj&ZMqCklh|ZGqSpR1h{tf%@y0+XI2nKd_fo1@*J4z@8CJ0{d8dHqW!}1*+hO-wEn}dwg;N^h*r(pwNlOUtnh*Gf(gPKL4Dzo-eJl$j699G zzuG3&BVOMF^*vDE13$P2jPG|-cHqs?*AqrskODd$ZDrTf*dibr%2q}=+0QROXHnq~6`;O;4#ZK99N znkvjYID5;wM*6_4S@*p(P5Ok(e4ejsI-`pJ>au+B;1M9->laGfrDa_DQ-yx5kjQ*N zg?`E`%U2au^1N9#AaI3S%0gnfC`-}AN0%2-NUJFyQ44sO_vohug=s=xpYj;9>uSEw z6_hIhUY=v3oZ~4Uv3bi?`uj?o*||B~Z-EA@Ij0`})=#oiiGi=i0Dj8OUoOI# z{&=wo=0EE8?~FDDN?=x-v1)ePLRH>wq8i^~u;0}^%auEKl7_miv2NZry>+3Q3EneX z7pP$w1Lbq<>^6l3bK8zvH^1G4oeSDe+`6zsQ36mm6@J58!tF&W%wDJ>>?OctWr>=m ztd1Ef(m4y5t86g~^xgO8)4|z``;69O0%x`v>jwAN=d~MOUEY48v9Lq2QIS((xN}N} zxN|0{wYf8We_kh_0|Mr%a9fcIk1SFVwn-}7UVJh5CxCxzt7CdL_&Xz=b1tyQx|Kk* zAo@6Y^oop=YIMrCuY1)M4ou;a_9c$yirpp|) zuFDLyzUwTty7LURywfzbIJZDKk>o{Tk*IE zd+8r6QB&Um_fNt7=8lnnP`>F96gKF9lZY$RP9v(UIB0g^e_pj)8!|Ir6 z(B}qt8@8B*hCQ|dSZt7!^*M617DmOdXiP4CHOCyUz3BJP%zoFnzqh30 zmJt~}&x5yHa0Vp5ym{6H`QyI;_bm?B`lhe;w)?^FH^A?jXXv{x;F)4~hf)RKDDcsI z0d8td*XioU{;SpQeyh|?+^@}Bmb|0Sk{hn-RdIwi7@NAyGkDJ6H=6Hc>_7tp-ZUm> zk2H$2hCDYeeb9u0wEh>2fexcn`)YoVu$5TB{RP22(m9(xpKrh`(%&zT|Mn{I-p08Y zf8ue#0Z?|z}&xN`TYrD*p@nSFz40$j8V@zu`+L+X0m@z(M z@R#7feN;-`C8JaFl*K+pWKAD<<_qa}6ZZ<~^S^+5g55cf{LXVY6Lv#W&hHt{nQSlC zoY9ta#qk!YYnMXjm!+?yK5)N*i3wgVtHE79NWTm0>A%{ac1+znVC}~@4_s$(H=u_B zEp!{8!xtO6&N<3>ISwBgi-ZQy)}+A*10FKqA7gY%-_=7Cd!EM_MNXDY6MjF1`?9ad zEj?1_8fJ_}#c-Bo!Un~;Qk+G(y#;4Z{=<9TXkyB?_e{7ysHyuWgiAOFL#&GtHp|wA}gi8;N1`DyN=Ya)_0k!Hg}t? zuFG2r{;LGKfd9Mu2d>w+-#KKfaYz1^7j7A}VKn2R>5kq@v`hkjCBK*P15F#wZ$B{? z9(uB@)#dQeaAQKoU<3X)Mke>!MxV=^G>f^(h>BezxXU>&oEyA}^P@ROJauOcAH=0z zBj>!UxRh%-FMiKFpgKNnw~?NHO?6rOS=GzIpME#s-$yreogKw^R9kzLD|lQW0zUZ* z_#0mQf0w_d@{S=}llKqWp!PC0o*I9bx`Aiqa(9}d7Pc=^i*iczJTSY>*w)iqUH;DG z>=DMq%zT3}U@!(AkvSC^IRg23lCzEU88q<8mz-JHAK+~I>-GD*f8x__PzmWbsf3K1 zLle?({%b<|UL!v3Mq@zBHE%EPH08M9FEV6fx491^Gt;g_w%?Su6&fsG0go6r_FrQN z--GjC;GNjp^Eazo2X0Wm8@$O|MXfu$ul1=pZ3-KK|GgrI^cYB&F_3o@h$H5<7{o^%B;O{wRPM`m6&;}O3b)*a$?3U)wIJ%PT%uPdG4%T z;Qyrue`8Dc@>h{5qez}nn>1J|nuhHh8CqYdA5&2MYs zk3QIVdh0O{OcDIEh8TtE17!@Xl5-^Nu}dyuy#8L!|B*8v{eX_CZ9PpnnGVwU_{cYsD zmNC)>_-OI=o=d#)NL}5>UGr}5hvm6b6myscp#k$_4`k3sf`2h%82tZ_G0@V^S;?G} z%^7)Xis<{3a;~MEV-D^{LdL#*;CNATivubp`#zP{>S6u*%*?A*a@L)|-Tp~gckbaC z13DP4jQ#UvjR5`wk%7-_W^8>)GhHtav(OuMyFMW+P{ z{B4Zi<hdeKGrS1NNz6Aff!2bawt<@t2c)y>~_KE!&Z67ND|4i_YOl^5Em^LIq zgBPW(jOmZTy_MiEvQy+ZIH}w6mkDn~(Z5BEhkKZ#4g=MqF9-;IWUTPKvA)Y}gZV@2 z52Aa^{42T@^VebUUk3jKF$btg*~8VujKSXF0{+ZRDLa8(4P0qAUJgIMQG23bR{f`D?!I1x@wtP_d%b@?E zm5gEJP{zJ#f`3BV%{6@^AKI68QZaM0RGTR|YV4>HDqx`{h%S?oYQ5Q!Fjgi%LtzuZ-kY`8KEVyCqgGiKNdJKs#W_T9aL$~ zNF_3KYW8SfRKM|aYStcbzenVwmYb<9?`x3S@{pBr+yj_Fd$uwz_NKJB_YV60HhAZ6 zB7bDeBe%A|d&-r5qp~d`*Vn=%MGi;7{{r&5jRl_7Ilm;&mnXlM&iA9w=QQ&I$o_BH zC(7PI$xG*AKbYQM{*4HK!xxFIA5dv64=VUXfj985hV<-5RZ7+Y|KzL#=Vj--A-u*s zykE;)#?Dm<%-3OeV@%|1GjdT1{x0BV;0wT;lTz|gzD%yN>!HO=a@v%V)8q>BTf8N? zlVz{7?0E$DYr_hh=dl;=8?gNT|Bmi)pUO;oP^GlGU(*HNK@Wo0Gxkpx9Rl2e-B%>0 z?pO9jX)0`Ph9CI%1)kSAHO)MmC;9wO&TX4YTWofBN&`6~5&X#mO|Fha2Qmp=S<9qrwa$6^kI@-4pP(vgRMDVerei&$q(Fx|7*VDEFxv+>^}0x0Fom@!z9T!A$!U7{?o$fV&Neyzo(<{ zwQ@|{M&`0D3Oz_==e{lSG!C4a#HZ{+@4u$z`B>5G()Iv1Hlok(j!zZbSq~&%J1OWD zr!-~S*%xHi%Z#26r05S0RAy4+f;J$HocAkU-awi5`ed8C%EtU zJT`MnZ<{-D0r-DWgZql~zjF4mNV&>r{7L)aVzIZlyQ#G$(l1J)@1H>uE{u0&LB{36DhKTlKyYrzw3A6??fLJyAHbaaMoeR<;-2tscFZHtP6&74+Hn6 zfRi>Y1CD~X5u3QpaQ2w;7IN}q`d;Tehi`5J{}B3GasFutXQMnN=cIVfM0pS_E6NaGz)ED-byvkd*0Myu6f=! z=eo(ce(SSl(`Y!$Z5!3=WaK=~xF`Q8{r?IwCk>gS97{N-792UFOrQm4)_nx%Gs^k@ zJ&v`R0p2r7^;xrX5-+6x2ZR4X@c%b5v>l+N|G)3$jN`m#m(T#)N!;>CKx|ka$=Y7l zn6%FWv?+ewoA#b#p5$zf6EdGNCmlzwkBwx`bs!_{^uL_9%=vl(ft<1WtemwcbfEuV zVN6&6jhe<^Xn-t5{x?PbByv8gKcK8tFn&d^{JG6Z~Ba}dCB)cgC?TuM3E=ldwKf5G3UfR$Y+y()@6Q#49eil5btB0 z?@Hb&@J0U1nP!}AVM5IVXYqkGJ^k;6WttwkEwM`*+vAorjO^E6MNEnAjI2H?dFs$( zj-dnXirT(uJB3Z?VcRJmGS+RjByR*UPES)(?8yUuCFXVLrzso zXNzJFpujNpgPOSqC0{^6^y`d^a@KwQnKR4ar^k^|)p~r$n##-(^%eh{49M9NcCa^V0(-N*)Hy$-9g^q6vyZC zUXv^NT&?$;dH13EKu z1b=djNuIG^=6+B1wy|$P_A{7mVgHgz7vV4Vuv`j+`~-WKh%q8UNlX)NeXk0T04tWH zekwpU)l1Wc%HPxTezho2X_jhYG~K$KYyMXPj~dFyY1pyk&Xk-Z_fVzU$-S}sId!=` zAH8=Xp(JogRVm(2{lt|KI;hlfd5#zA@IUTG6=nW^BGSAcX+q2s7D?a)FvStxLW=PRn8`RTc~JLEIgVu_`2_c1(=Kyx{B6He>GR9N?h&3TjCc-|+j7^fbq-f#;@S#dvk3FWs6Ax3M@Dns3 z7O4ky-${JH>%<3qh%aP5{shHmT{(VUwSqW-RmANq>4?9Um>v8)stg!7?*tfx6bClePq3@wsq{MR(6~wZ7<|qAJJBc#S!25abhCY zE+!6b&1EySzYO1v_L)fx4!AYq+4E<$9($hj9q}soH}ZANO<~$V-R`i+3B(7Lth7W; z`AEmt;Ik*T<|J)@o%Q8P>W|3Fm@RduQ1^d`8L!sK%ZISkA z6FU=tUnZCMn$_TT4>(lft2lt4cQrniPWagy7p4zT6ElbCcraqCCerQ`W}HSj@g?q$ zsP_X|mz!(yb&pu%H%HHFTR4}P1Bst{A0JLfK=FPZ2eY^TTKt}C#7`cN4{;BD@QL_- z#OEtXLrBVI)()weh1<{{wcn2@yFsvtKQsw{wMgIJ|^Dag!r4v zh*uFmF?Ii8WJ;cc_!OJfG3zBAXArma6V~q?Sj%fW1$GtjKlQ=)_fBl`j_Ls|-2WAt z6Myjxez+j|T~UYj_flsD{`VJeC3b`OjT7K;h<+%vy8A?n^zFu+4%3Onn05jlpTD(x zc>}4Rwl*h@(6-UZ*S zAG{if&u|>^F8@dU5^M1UW6&hf_|K+pWdsTcTeq;RJNAM#*g->}O?ca>Qc>_N0J;Z%I zP5ocV7@X2_SN#iq)s^`6M&oPh4_*g7 z{_Hu48MkR)u=Fty3J$}d#h&dw7QL{o$D&W+P3`wx*O|DtPGwqFBCp?r|8~;;_)_iX z9ir^%*z8k?-yt4`zQ2_AEyB0>ruY+0zv!d*sef;zw0wY=fdkd1KXLyyo>7r0x2fxL zm#NJ?=c}TTm#f7+Cj0jt+owaXf}Xp2UEbrpUL$)Lg9hh)R?>6S?G+tL3aLMp_BWKc zP0=E}ahq}YXW?%RE&Cr4A5>Cm%ZJLr;gH9l`~m+vLjQl}@dw_5U&KGj@v!#Q*g3Zo zTSY8B%(d3JiLY5^C!CDHcNsnnJ3D^Mu%eikn@^1Dk>5K{O>RF*!2@b~tFbk6kbFpL zxmS2h$D~Nxv`??)!SnG+M&gr9O>S{-25p}%{DEKT0P#-eMa|5S`U8MM;3cu~Vki9! zJRZenw*$M^t*nbblr<(fM_*)Z6T>=MuMuih?kNZ#VE2X|JN3Au?f4iE(H{@U_>=w_ zNZsO#yE97}4e+rAVYk_az3I5;9Mgxe=Uj%Z{Gt%*!8YQL?P?hIpO3{BMBTH942cIU$qJweC zJB^rj%Qx8ENe6V@%hUgX59uEBeCN!MxnCxBTWrwUMied)!eK`<%L zp}yAu&Kl3U0o%R5_^*fq5dUX1{qS$$0qj3>waxxa_3Kz8vCXe`F%LZl9^xCpFS7YX z=HOSEgHLidqWXpD{0qgFk3YrhLupIY2i-^JL%_TV~?+2@p}SOc`PcdWbUr*&8T+AvciMvcQEYDSK+kv?Th=)VDjdJG^Z|8E0nGC* z7o|+bmaC%b=saT?uh?U|Fn+%Phd1a;8~lWi0vjg%fV>Mv?mZ63nR>^dPgejxOX``L zbJqXUHr3J2mgM>T1ROqR9QP0&=B^^9x-P&+zY$tjBX8wQpQk7XiLa@yF6>qcJ5MM! zlIOJDsv!Ca`eXq0_eQSmqd&e9pQw3{8!x;jw$i$~v^{VnHnn{FGRb!)HZ<_~5?lEf z;Ni)K%za$U4(xvy$QZAyGgYMx+O}fm@wAY)QSvRBxm&$9y%CP1*u1x5FKz~()Vcjm z@_p$1K4R1AYjt4%*@FE?@<=|1UG5h8B~RP1M(K0sRP#bveb?=!?NR-);gGZFl1^a_ z$Ws$cj+$`pjU?}s^pU{X=9{a+OoP$HYl->ZNV_pBRI*F03XuD05WxRszim^)EhZoV z_UqLErV2Mu8DL0HVV2OgiW)mb!>`*ZWIu}RSqifj`lACOA6K#`Y=W$b9q4t-h^d{) z8mpmK=a)S_vR6!E3J1h@W-rTx3)qt|iMaSX*h8_^7Plg6PMa~otO*}vP5C6hoyywU zA3ZhzeZ4*93Fv7<*xO($LQiImI*Q*O<+t}UAKc_f*z|P&tnzIY9g1IKZ56w&%WTE^ zFPyyyH=u`qK%DyrtoM$hWB#ikZNNa{@HY_0{(-}_CL!E5QN<+hENhdo<>-Q(>4(>K znV!wM<$U7(_Y=F#`gQdXbbI!N757C~eu=f>tIS7J(A7U-jnYkY)%c8CFG|d~`S)Yn zt@y{LZnGEV3-|32TY#P3E1Ud3oJcLF<7M zN361PU*J94vBW7q9gCNCW3DT8&%vk}9S7K#q`p90!}I6`DV8}27cx(+MGifL%x)~Q zT7K`0U)=~D=1O#{S?E>PxTGCdM0cM5n%$lDq+><;dgjnq(4RVs-w~Z{4zl%OWL#t9 zn#8?GEVjhJ4B_(&(XNdzJ2RKPYcK9})V?@rV*GV&8^&zOz6bqdtN6W{ABQ4`^F;@f zc|z8iGOx?UJ=PJuu8Vb4?gr+K2hf)?^Dj^k-S@S gVUNzmj=miEIS{$tSmrI6KYr}x78`GX2e8xrKTq Date: Mon, 19 Aug 2024 17:30:14 +0200 Subject: [PATCH 04/13] Rephrase warnings --- docs/src/dev/api.md | 2 +- docs/src/dev/how_it_works.md | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/src/dev/api.md b/docs/src/dev/api.md index 954245d8..ae4d000b 100644 --- a/docs/src/dev/api.md +++ b/docs/src/dev/api.md @@ -1,7 +1,7 @@ # [Internals Reference](@id internal-api) !!! warning "Internals may change" - This part of the developer documentation exclusively refers to internals that may change without warning in a future release of SparseConnectivityTracer. + This part of the developer documentation **exclusively** refers to internals that may change without warning in a future release of SparseConnectivityTracer. Anything written on this page should be treated as if it was undocumented. Only functionality that is exported or part of the [user documentation](@ref api) adheres to semantic versioning. diff --git a/docs/src/dev/how_it_works.md b/docs/src/dev/how_it_works.md index 2bc681ac..f0a1e63b 100644 --- a/docs/src/dev/how_it_works.md +++ b/docs/src/dev/how_it_works.md @@ -1,7 +1,8 @@ # How SparseConnectivityTracer works !!! warning "Internals may change" - The developer documentation might refer to internals that may change without warning in a future release of SparseConnectivityTracer. + The developer documentation refers to internals that may change without warning in a future release of SparseConnectivityTracer. + Anything written on this page should be treated as if it was undocumented. Only functionality that is exported or part of the [user documentation](@ref api) adheres to semantic versioning. From d5c6f78085d42c983c3283020d9c0245db28a39c Mon Sep 17 00:00:00 2001 From: adrhill Date: Mon, 19 Aug 2024 19:16:50 +0200 Subject: [PATCH 05/13] Update "How SCT works" --- docs/Project.toml | 1 + docs/make.jl | 1 + docs/src/dev/how_it_works.md | 127 +++++++++++++++++++++++++++++++++-- 3 files changed, 122 insertions(+), 7 deletions(-) diff --git a/docs/Project.toml b/docs/Project.toml index 46474468..1384aa68 100644 --- a/docs/Project.toml +++ b/docs/Project.toml @@ -1,6 +1,7 @@ [deps] ADTypes = "47edcb42-4c32-4615-8424-f2b9edc5f35b" Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4" +DocumenterMermaid = "a078cd44-4d9c-4618-b545-3ab9d77f9177" SparseConnectivityTracer = "9f842d2f-2579-4b1d-911e-f412cf18a3f5" [compat] diff --git a/docs/make.jl b/docs/make.jl index 3677d206..8ada095f 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -1,5 +1,6 @@ using SparseConnectivityTracer using Documenter +using DocumenterMermaid # Create index.md from README cp(joinpath(@__DIR__, "..", "README.md"), joinpath(@__DIR__, "src", "index.md"); force=true) diff --git a/docs/src/dev/how_it_works.md b/docs/src/dev/how_it_works.md index f0a1e63b..ac2b3336 100644 --- a/docs/src/dev/how_it_works.md +++ b/docs/src/dev/how_it_works.md @@ -1,12 +1,11 @@ # How SparseConnectivityTracer works !!! warning "Internals may change" - The developer documentation refers to internals that may change without warning in a future release of SparseConnectivityTracer. - Anything written on this page should be treated as if it was undocumented. + The developer documentation might refer to internals which can change without warning in a future release of SparseConnectivityTracer. Only functionality that is exported or part of the [user documentation](@ref api) adheres to semantic versioning. -SparseConnectivityTracer works by pushing `Real` number types called tracers through generic functions. +SparseConnectivityTracer (SCT) works by pushing `Real` number types called tracers through generic functions using operator overloading. Currently, two tracer types are provided: * [`GradientTracer`](@ref SparseConnectivityTracer.GradientTracer): used for Jacobian sparsity patterns @@ -22,11 +21,125 @@ This is how [**local** spasity patterns](@ref TracerLocalSparsityDetector) are c This is a good mental model for SparseConnectivityTracer if you are already familiar with ForwardDiff and its limitations. -## Index Sets +## Index sets Let's take a look at a scalar function $f: \mathbb{R}^n \rightarrow \mathbb{R}$. -The gradient is defined as the vector $\frac{\partial f}{\partial x_i}$ -and the Hessian as the matrix $\frac{\partial^2 f}{\partial x_i \partial x_j}$ for a given input $x\in\mathbb{R}^n$. +For a given input $\mathbf{x} \in \mathbb{R}^n$, +the gradient of $f$ is defined as $\left(\nabla f(\mathbf{x})\right)_{i} = \frac{\partial f}{\partial x_i}$ +and the Hessian as $\left(\nabla^2 f(\mathbf{x})\right)_{i,j} = \frac{\partial^2 f}{\partial x_i \partial x_j}$. +Sparsity patterns correspond to the mask of non-zero values in the gradient and Hessian. +Instead of saving the values of individual partial derivatives, they can efficiently be represented by the set of indices correponding to non-zero values: + +* Gradient patterns are represented by sets of indices $\left\{i \;\big|\; \left(\nabla f(\mathbf{x})\right)_{i} \neq 1\right\}$ +* Hessian patterns are represented by sets of index tuples $\left\{(i, j) \;\Big|\; \left(\nabla^2 f(\mathbf{x})\right)_{i,j} \neq 1\right\}$ + +## Motivating example + +Let's take a look at the computational graph of the equation $f(\mathbf{x}) = x_1 + x_2x_3 + \text{sgn}(x_4)$, +where $\text{sgn}$ is the [sign function](https://en.wikipedia.org/wiki/Sign_function): + + +```mermaid +flowchart LR + subgraph Inputs + X1["$$x_1$$"] + X2["$$x_2$$"] + X3["$$x_3$$"] + X4["$$x_4$$"] + end + + PLUS((+)) + TIMES((*)) + SIGN((sgn)) + PLUS2((+)) + + X1 --> |"{1}"| PLUS + X2 --> |"{2}"| TIMES + X3 --> |"{3}"| TIMES + X4 --> |"{4}"| SIGN + TIMES --> |"{2,3}"| PLUS + PLUS --> |"{1,2,3}"| PLUS2 + SIGN --> |"{}"| PLUS2 + + PLUS2 --> |"{1,2,3}"| RES["$$y=f(x)$$"] +``` +To obtain a sparsity pattern, each scalar input $x_i$ gets seeded with a corresponding singleton index set $\{i\}$ [^1]. +Since addition and multiplication have non-zero derivatives with respect to both of their inputs, +the resulting scalar values accumulate and propagate their index sets (annotated on the edged of the graph). +The sign function has zero derivatives for any input value. It therefore doesn't propagate the index set ${4}$ corresponding to the input $x_4$. + +[^1]: since $\frac{\partial x_i}{\partial x_j} \neq 0$ iff $i \neq j$ + +The resulting **global** gradient sparsity pattern $\left(\nabla f(\mathbf{x})\right)_{i} \neq 1$ for $i$ in $\{1, 2, 3\}$ matches the analytical gradient + +```math +\nabla f(\mathbf{x}) = \begin{bmatrix} + \frac{\partial f}{\partial x_1} \\ + \frac{\partial f}{\partial x_2} \\ + \frac{\partial f}{\partial x_3} \\ + \frac{\partial f}{\partial x_4} +\end{bmatrix} += +\begin{bmatrix} + 1 \\ + x_3 \\ + x_2 \\ + 0 +\end{bmatrix} \quad . +``` + +Note that the **local** sparsity pattern could be more sparse in case $x_3$ and/or $x_2$ are zero. +Computing such local sparsity patterns requires [`Dual`](@ref SparseConnectivityTracer.Dual) numbers with information about the primal computation. +These can be used to evaluate the **local** differentiability of operations like multiplication. + +## Toy implementation + +As mentioned above, SCT uses operator overloading to keep track of index sets. +Let's start by implementing our own `MyGradientTracer` type: + +```@example toytracer +struct MyGradientTracer + indexset::Set +end +``` + +We can now overload operators from Julia Base using our type: + +```@example toytracer +import Base: +, *, sign + +Base.:+(a::MyGradientTracer, b::MyGradientTracer) = MyGradientTracer(union(a.indexset, b.indexset)) +Base.:*(a::MyGradientTracer, b::MyGradientTracer) = MyGradientTracer(union(a.indexset, b.indexset)) +Base.sign(x::MyGradientTracer) = MyGradientTracer(Set()) # return empty index set +``` + +Let's create a vector of tracers to represent our input and evaluate our function with it: + +```@example toytracer +f(x) = x[1] + x[2]*x[3] * sign(x[4]) + +xtracer = [ + MyGradientTracer(Set(1)), + MyGradientTracer(Set(2)), + MyGradientTracer(Set(3)), + MyGradientTracer(Set(4)), +] + +ytracer = f(xtracer) +``` + +Compared to this toy implementation, SCT adds some utilities to automatically create `xtracer` and parse the output `ytracer` into a sparse matrix, which we will omit here. + +[`jacobian_sparsity(f, x, TracerSparsityDetector())`](@ref TracerSparsityDetector) calls these three steps of (1) tracer creation, (2) function evaluation and (3) output parsing in sequence: + +```@example toytracer +using SparseConnectivityTracer + +x = rand(4) +jacobian_sparsity(f, x, TracerSparsityDetector()) +``` + +! tip "From gradients to Jacobians" + -## Operator overloading: Toy example \ No newline at end of file From 527770ef6fa7dff1760800891918b75b8ddc1cde Mon Sep 17 00:00:00 2001 From: adrhill Date: Mon, 19 Aug 2024 19:17:04 +0200 Subject: [PATCH 06/13] Remove ADTypes import from docstrings --- src/adtypes.jl | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/adtypes.jl b/src/adtypes.jl index a47b4c56..30f6797f 100644 --- a/src/adtypes.jl +++ b/src/adtypes.jl @@ -9,7 +9,7 @@ For local sparsity patterns at a specific input point, use [`TracerLocalSparsity # Example ```jldoctest -julia> using ADTypes, SparseConnectivityTracer +julia> using SparseConnectivityTracer julia> ADTypes.jacobian_sparsity(diff, rand(4), TracerSparsityDetector()) 3×4 SparseArrays.SparseMatrixCSC{Bool, Int64} with 6 stored entries: @@ -19,7 +19,7 @@ julia> ADTypes.jacobian_sparsity(diff, rand(4), TracerSparsityDetector()) ``` ```jldoctest -julia> using ADTypes, SparseConnectivityTracer +julia> using SparseConnectivityTracer julia> f(x) = x[1] + x[2]*x[3] + 1/x[4]; @@ -68,7 +68,7 @@ For global sparsity patterns, use [`TracerSparsityDetector`](@ref). # Example ```jldoctest -julia> using ADTypes, SparseConnectivityTracer +julia> using SparseConnectivityTracer julia> f(x) = x[1] > x[2] ? x[1:3] : x[2:4]; @@ -86,7 +86,7 @@ julia> jacobian_sparsity(f, [2.0, 1.0, 3.0, 4.0], TracerLocalSparsityDetector()) ``` ```jldoctest -julia> using ADTypes, SparseConnectivityTracer +julia> using SparseConnectivityTracer julia> f(x) = x[1] + max(x[2], x[3]) * x[3] + 1/x[4]; From fc8bb5cbef4d04c911e0317ef669e154255a5f90 Mon Sep 17 00:00:00 2001 From: adrhill Date: Mon, 19 Aug 2024 20:09:18 +0200 Subject: [PATCH 07/13] Finalize "How it works" guide --- docs/src/dev/api.md | 2 +- docs/src/dev/how_it_works.md | 66 +++++++++++++++++++++++++++++------- src/adtypes.jl | 2 +- 3 files changed, 56 insertions(+), 14 deletions(-) diff --git a/docs/src/dev/api.md b/docs/src/dev/api.md index ae4d000b..d9eee9af 100644 --- a/docs/src/dev/api.md +++ b/docs/src/dev/api.md @@ -1,6 +1,6 @@ # [Internals Reference](@id internal-api) -!!! warning "Internals may change" +!!! danger "Internals may change" This part of the developer documentation **exclusively** refers to internals that may change without warning in a future release of SparseConnectivityTracer. Anything written on this page should be treated as if it was undocumented. Only functionality that is exported or part of the [user documentation](@ref api) adheres to semantic versioning. diff --git a/docs/src/dev/how_it_works.md b/docs/src/dev/how_it_works.md index ac2b3336..8f46356a 100644 --- a/docs/src/dev/how_it_works.md +++ b/docs/src/dev/how_it_works.md @@ -1,10 +1,12 @@ # How SparseConnectivityTracer works -!!! warning "Internals may change" +!!! danger "Internals may change" The developer documentation might refer to internals which can change without warning in a future release of SparseConnectivityTracer. Only functionality that is exported or part of the [user documentation](@ref api) adheres to semantic versioning. +## Tracers are scalars + SparseConnectivityTracer (SCT) works by pushing `Real` number types called tracers through generic functions using operator overloading. Currently, two tracer types are provided: @@ -16,9 +18,9 @@ Alternatively, these can be used inside of a dual number type [`Dual`](@ref Spar which keeps track of the primal computation and allows tracing through comparisons and control flow. This is how [**local** spasity patterns](@ref TracerLocalSparsityDetector) are computed. -!!! tip "Tip: SparseConnectivityTracer as binary ForwardDiff" +!!! tip "Tip: View SparseConnectivityTracer as binary ForwardDiff" SparseConnectivityTracer's `Dual{T, GradientTracer}` can be thought of as a binary version of [ForwardDiff](https://github.com/JuliaDiff/ForwardDiff.jl)'s own `Dual` number type. - This is a good mental model for SparseConnectivityTracer if you are already familiar with ForwardDiff and its limitations. + This is a good mental model for SparseConnectivityTracer if you are familiar with ForwardDiff and its limitations. ## Index sets @@ -31,8 +33,15 @@ and the Hessian as $\left(\nabla^2 f(\mathbf{x})\right)_{i,j} = \frac{\partial^2 Sparsity patterns correspond to the mask of non-zero values in the gradient and Hessian. Instead of saving the values of individual partial derivatives, they can efficiently be represented by the set of indices correponding to non-zero values: -* Gradient patterns are represented by sets of indices $\left\{i \;\big|\; \left(\nabla f(\mathbf{x})\right)_{i} \neq 1\right\}$ -* Hessian patterns are represented by sets of index tuples $\left\{(i, j) \;\Big|\; \left(\nabla^2 f(\mathbf{x})\right)_{i,j} \neq 1\right\}$ +* Gradient patterns are represented by sets of indices $\left\{i \;\big|\; \left(\nabla f(\mathbf{x})\right)_{i} \neq 1\right\}$ +* Local Hessian patterns are represented by sets of index tuples $\left\{(i, j) \;\Big|\; \left(\nabla^2 f(\mathbf{x})\right)_{i,j} \neq 1\right\}$ + + +!!! warning "Global vs. Local" + Global sparsity patterns are the index sets over all $\mathbf{x}\in\mathbb{R}^n$, + whereas local patterns are the index sets for a given point $\mathbf{x}$. + For a given function $f$, global sparsity patterns are therefore always supersets of local sparsity patterns + and more "conservative" in the sense that they are less sparse. ## Motivating example @@ -66,8 +75,8 @@ flowchart LR ``` To obtain a sparsity pattern, each scalar input $x_i$ gets seeded with a corresponding singleton index set $\{i\}$ [^1]. Since addition and multiplication have non-zero derivatives with respect to both of their inputs, -the resulting scalar values accumulate and propagate their index sets (annotated on the edged of the graph). -The sign function has zero derivatives for any input value. It therefore doesn't propagate the index set ${4}$ corresponding to the input $x_4$. +the resulting values accumulate and propagate their index sets (annotated on the edges of the graph above). +The sign function has zero derivatives for any input value. It therefore doesn't propagate the index set ${4}$ corresponding to the input $x_4$. Instead, it returns an empty set. [^1]: since $\frac{\partial x_i}{\partial x_j} \neq 0$ iff $i \neq j$ @@ -89,9 +98,10 @@ The resulting **global** gradient sparsity pattern $\left(\nabla f(\mathbf{x})\r \end{bmatrix} \quad . ``` -Note that the **local** sparsity pattern could be more sparse in case $x_3$ and/or $x_2$ are zero. -Computing such local sparsity patterns requires [`Dual`](@ref SparseConnectivityTracer.Dual) numbers with information about the primal computation. -These can be used to evaluate the **local** differentiability of operations like multiplication. +!!! tip "From Global to Local" + Note that the **local** sparsity pattern could be more sparse in case $x_2$ and/or $x_3$ are zero. + Computing such local sparsity patterns requires [`Dual`](@ref SparseConnectivityTracer.Dual) numbers with information about the primal computation. + These are used to evaluate the **local** differentiability of operations like multiplication. ## Toy implementation @@ -140,6 +150,38 @@ x = rand(4) jacobian_sparsity(f, x, TracerSparsityDetector()) ``` -! tip "From gradients to Jacobians" - +## Tracing Jacobians + +Our toy implementation above doesn't just work on scalar functions, but also on vector valued functions: + +```@example toytracer +g(x) = [x[1], x[2]*x[3], x[1]+x[4]] +g(xtracer) +``` + +By stacking individual `MyGradientTracer`s row-wise, we obtain the sparsity pattern of the Jacobian of $g$ + +```math +J_g(\mathbf{x})= +\begin{pmatrix} +1 & 0 & 0 & 0 \\ +0 & x_3 & x_2 & 0 \\ +1 & 0 & 0 & 1 +\end{pmatrix} \quad . +``` + +We obtain the same result using SCT's `jacobian_sparsity`: +```@example toytracer +jacobian_sparsity(g, x, TracerSparsityDetector()) +``` + +## Tracing Hessians + +In the sections above, we outlined how to implement our own [`GradientTracer`](@ref SparseConnectivityTracer.GradientTracer) from scratch. +[`HessianTracer`](@ref SparseConnectivityTracer.HessianTracer) use the same operator overloading approach but are a bit more involved as they contain two index sets: +one for the gradient pattern and one for the Hessian pattern. +These sets are updated based on whether the first- and second-order derivatives of an operator are zero or not. +!!! tip "To be published" + Look forward to our upcomping publication of SparseConnectivityTracer, + where we will go into more detail on the implementation of `HessianTracer`! diff --git a/src/adtypes.jl b/src/adtypes.jl index 30f6797f..6442db16 100644 --- a/src/adtypes.jl +++ b/src/adtypes.jl @@ -11,7 +11,7 @@ For local sparsity patterns at a specific input point, use [`TracerLocalSparsity ```jldoctest julia> using SparseConnectivityTracer -julia> ADTypes.jacobian_sparsity(diff, rand(4), TracerSparsityDetector()) +julia> jacobian_sparsity(diff, rand(4), TracerSparsityDetector()) 3×4 SparseArrays.SparseMatrixCSC{Bool, Int64} with 6 stored entries: 1 1 ⋅ ⋅ ⋅ 1 1 ⋅ From 34e3144879cd693fa3a068c2048129eaa2fd019a Mon Sep 17 00:00:00 2001 From: adrhill Date: Mon, 19 Aug 2024 20:12:30 +0200 Subject: [PATCH 08/13] Fix typos --- README.md | 2 +- docs/src/dev/how_it_works.md | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 202227c7..f9a45ebd 100644 --- a/README.md +++ b/README.md @@ -97,7 +97,7 @@ julia> hessian_sparsity(g, x, detector) ⋅ 1 ⋅ ⋅ 1 ``` -For more detailled examples, take a look at the [documentation](https://adrianhill.de/SparseConnectivityTracer.jl/dev). +For more detailed examples, take a look at the [documentation](https://adrianhill.de/SparseConnectivityTracer.jl/dev). ### Local tracing diff --git a/docs/src/dev/how_it_works.md b/docs/src/dev/how_it_works.md index 8f46356a..e921a5a3 100644 --- a/docs/src/dev/how_it_works.md +++ b/docs/src/dev/how_it_works.md @@ -16,7 +16,7 @@ Currently, two tracer types are provided: When used alone, these tracers compute [**global** sparsity patterns](@ref TracerSparsityDetector). Alternatively, these can be used inside of a dual number type [`Dual`](@ref SparseConnectivityTracer.Dual), which keeps track of the primal computation and allows tracing through comparisons and control flow. -This is how [**local** spasity patterns](@ref TracerLocalSparsityDetector) are computed. +This is how [**local** sparsity patterns](@ref TracerLocalSparsityDetector) are computed. !!! tip "Tip: View SparseConnectivityTracer as binary ForwardDiff" SparseConnectivityTracer's `Dual{T, GradientTracer}` can be thought of as a binary version of [ForwardDiff](https://github.com/JuliaDiff/ForwardDiff.jl)'s own `Dual` number type. @@ -31,7 +31,7 @@ the gradient of $f$ is defined as $\left(\nabla f(\mathbf{x})\right)_{i} = \frac and the Hessian as $\left(\nabla^2 f(\mathbf{x})\right)_{i,j} = \frac{\partial^2 f}{\partial x_i \partial x_j}$. Sparsity patterns correspond to the mask of non-zero values in the gradient and Hessian. -Instead of saving the values of individual partial derivatives, they can efficiently be represented by the set of indices correponding to non-zero values: +Instead of saving the values of individual partial derivatives, they can efficiently be represented by the set of indices corresponding to non-zero values: * Gradient patterns are represented by sets of indices $\left\{i \;\big|\; \left(\nabla f(\mathbf{x})\right)_{i} \neq 1\right\}$ * Local Hessian patterns are represented by sets of index tuples $\left\{(i, j) \;\Big|\; \left(\nabla^2 f(\mathbf{x})\right)_{i,j} \neq 1\right\}$ @@ -183,5 +183,5 @@ one for the gradient pattern and one for the Hessian pattern. These sets are updated based on whether the first- and second-order derivatives of an operator are zero or not. !!! tip "To be published" - Look forward to our upcomping publication of SparseConnectivityTracer, + Look forward to our upcoming publication of SparseConnectivityTracer, where we will go into more detail on the implementation of `HessianTracer`! From 95db15f6c644687f75f17af5d0a41ca665b4938e Mon Sep 17 00:00:00 2001 From: adrhill Date: Tue, 20 Aug 2024 16:06:38 +0200 Subject: [PATCH 09/13] Improve docstrings --- src/adtypes.jl | 35 +++++++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 6 deletions(-) diff --git a/src/adtypes.jl b/src/adtypes.jl index 6442db16..3453c5c6 100644 --- a/src/adtypes.jl +++ b/src/adtypes.jl @@ -19,8 +19,6 @@ julia> jacobian_sparsity(diff, rand(4), TracerSparsityDetector()) ``` ```jldoctest -julia> using SparseConnectivityTracer - julia> f(x) = x[1] + x[2]*x[3] + 1/x[4]; julia> hessian_sparsity(f, rand(4), TracerSparsityDetector()) @@ -67,18 +65,45 @@ For global sparsity patterns, use [`TracerSparsityDetector`](@ref). # Example +Local sparsity patterns are less convervative than global patterns and need to be recomputed for each input `x`: + ```jldoctest julia> using SparseConnectivityTracer +julia> method = TracerLocalSparsityDetector(); + +julia> f(x) = x[1] * x[2]; # J_f = [x[2], x[1]] + +julia> jacobian_sparsity(f, [1, 0], method) +1×2 SparseArrays.SparseMatrixCSC{Bool, Int64} with 1 stored entry: + ⋅ 1 + +julia> jacobian_sparsity(f, [0, 1], method) +1×2 SparseArrays.SparseMatrixCSC{Bool, Int64} with 1 stored entry: + 1 ⋅ + +julia> jacobian_sparsity(f, [0, 0], method) +1×2 SparseArrays.SparseMatrixCSC{Bool, Int64} with 0 stored entries: + ⋅ ⋅ + +julia> jacobian_sparsity(f, [1, 1], method) +1×2 SparseArrays.SparseMatrixCSC{Bool, Int64} with 2 stored entries: + 1 1 +``` + +`TracerLocalSparsityDetector` can compute sparsity patterns of functions that contain comparisons and `ifelse` statements: + + +```jldoctest julia> f(x) = x[1] > x[2] ? x[1:3] : x[2:4]; -julia> jacobian_sparsity(f, [1.0, 2.0, 3.0, 4.0], TracerLocalSparsityDetector()) +julia> jacobian_sparsity(f, [1, 2, 3, 4], TracerLocalSparsityDetector()) 3×4 SparseArrays.SparseMatrixCSC{Bool, Int64} with 3 stored entries: ⋅ 1 ⋅ ⋅ ⋅ ⋅ 1 ⋅ ⋅ ⋅ ⋅ 1 -julia> jacobian_sparsity(f, [2.0, 1.0, 3.0, 4.0], TracerLocalSparsityDetector()) +julia> jacobian_sparsity(f, [2, 1, 3, 4], TracerLocalSparsityDetector()) 3×4 SparseArrays.SparseMatrixCSC{Bool, Int64} with 3 stored entries: 1 ⋅ ⋅ ⋅ ⋅ 1 ⋅ ⋅ @@ -86,8 +111,6 @@ julia> jacobian_sparsity(f, [2.0, 1.0, 3.0, 4.0], TracerLocalSparsityDetector()) ``` ```jldoctest -julia> using SparseConnectivityTracer - julia> f(x) = x[1] + max(x[2], x[3]) * x[3] + 1/x[4]; julia> hessian_sparsity(f, [1.0, 2.0, 3.0, 4.0], TracerLocalSparsityDetector()) From d1d58f7804db3e8c4fa3d92d3e94ee64a0e04973 Mon Sep 17 00:00:00 2001 From: adrhill Date: Tue, 20 Aug 2024 16:06:58 +0200 Subject: [PATCH 10/13] Minor fixes --- docs/src/dev/how_it_works.md | 4 ++-- docs/src/user/api.md | 6 +----- 2 files changed, 3 insertions(+), 7 deletions(-) diff --git a/docs/src/dev/how_it_works.md b/docs/src/dev/how_it_works.md index e921a5a3..38e909ba 100644 --- a/docs/src/dev/how_it_works.md +++ b/docs/src/dev/how_it_works.md @@ -75,10 +75,10 @@ flowchart LR ``` To obtain a sparsity pattern, each scalar input $x_i$ gets seeded with a corresponding singleton index set $\{i\}$ [^1]. Since addition and multiplication have non-zero derivatives with respect to both of their inputs, -the resulting values accumulate and propagate their index sets (annotated on the edges of the graph above). +their outputs accumulate and propagate the index sets of their inputs (annotated on the edges of the graph above). The sign function has zero derivatives for any input value. It therefore doesn't propagate the index set ${4}$ corresponding to the input $x_4$. Instead, it returns an empty set. -[^1]: since $\frac{\partial x_i}{\partial x_j} \neq 0$ iff $i \neq j$ +[^1]: $\frac{\partial x_i}{\partial x_j} \neq 0$ only holds for $i=j$ The resulting **global** gradient sparsity pattern $\left(\nabla f(\mathbf{x})\right)_{i} \neq 1$ for $i$ in $\{1, 2, 3\}$ matches the analytical gradient diff --git a/docs/src/user/api.md b/docs/src/user/api.md index 37f00569..a9d84470 100644 --- a/docs/src/user/api.md +++ b/docs/src/user/api.md @@ -5,12 +5,8 @@ CollapsedDocStrings = true ``` # [API Reference](@id api) -```@index -``` - -## ADTypes Interface -SparseConnectivityTracer uses [ADTypes.jl](https://github.com/SciML/ADTypes.jl)'s interface for [sparsity detection](https://sciml.github.io/ADTypes.jl/stable/#Sparsity-detector). +SparseConnectivityTracer uses [ADTypes.jl](https://github.com/SciML/ADTypes.jl)'s [interface for sparsity detection](https://sciml.github.io/ADTypes.jl/stable/#Sparsity-detector). In fact, the functions `jacobian_sparsity` and `hessian_sparsity` are re-exported from ADTypes. To compute **global** sparsity patterns of `f(x)` over the entire input domain `x`, use From e001833a1b0b6661d2adf716f75272f0a800129a Mon Sep 17 00:00:00 2001 From: adrhill Date: Tue, 20 Aug 2024 17:47:47 +0200 Subject: [PATCH 11/13] Document limitations --- docs/make.jl | 3 +- docs/src/user/limitations.md | 118 +++++++++++++++++++++++++++++++++++ 2 files changed, 120 insertions(+), 1 deletion(-) create mode 100644 docs/src/user/limitations.md diff --git a/docs/make.jl b/docs/make.jl index 8ada095f..1ecc9e38 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -23,7 +23,8 @@ makedocs(; ), pages=[ "Getting Started" => "index.md", - "User Documentation" => ["API Reference" => "user/api.md"], + "User Documentation" => + ["Limitations" => "user/limitations.md", "API Reference" => "user/api.md"], "Developer Documentation" => [ "How SCT works" => "dev/how_it_works.md", "Internals Reference" => "dev/api.md", diff --git a/docs/src/user/limitations.md b/docs/src/user/limitations.md new file mode 100644 index 00000000..60a363e9 --- /dev/null +++ b/docs/src/user/limitations.md @@ -0,0 +1,118 @@ +# [Limitations](@id limitations) + +## Sparsity patterns are conservative approximations + +Sparsity patterns returned by SparseConnectivityTracer (SCT) can in some cases be overly conservative, meaning that they might contain "too many ones". +If you observe an overly conservative pattern, [please open a feature request](https://github.com/adrhill/SparseConnectivityTracer.jl/issues) so we know where to add more method overloads. + +!!! warning "No-false-negatives policy" + If you ever observe a sparsity pattern that contains too many zeros, we urge you to [open a bug report](https://github.com/adrhill/SparseConnectivityTracer.jl/issues)! + +## Limited control flow + +[`TracerSparsityDetector`](@ref) does not support any control flow or boolean functions. +This might seem unintuitive but follows from our policy stated above: SCT guarantees conservative sparsity patterns. +Using an approach based on operator-overloading, this means that global sparsity detection isn't allowed to hit any branching code. + +Only [`TracerLocalSparsityDetector`](@ref) supports limited amounts of comparison operators and control flow, i.e.: +* comparison operators (`<`, `==`, ...) +* indicator functions (`iszero`, `iseven`, ...) +* `ifelse` statements + +!!! warning "Common control flow errors" + By design, SCT will throw errors instead of returning wrong sparsity patterns. Common error messages include: + + ```julia + ERROR: TypeError: non-boolean [tracer type] used in boolean context + ``` + + ```julia + ERROR: Function [function] requires primal value(s). + A dual-number tracer for local sparsity detection can be used via `TracerLocalSparsityDetector`. + ``` + +!!! details "Why does TracerSparsityDetector not support control flow and comparisons?" + Let us motivate the design decision above by a simple example function: + + ```@example ctrlflow + function f(x) + if x[1] > x[2] + return x[1] + else + return x[2] + end + end + nothing # hide + ``` + + The desired **global** Jacobian sparsity pattern over the entire input domain $x \in \mathbb{R}^2$ is `[1 1]`. + Two **local** sparsity patterns are possible: + `[1 0]` for $\{x | x_1 > x_2\}$, + `[0 1]` for $\{x | x_1 \le x_2\}$. + + The local sparsity patterns of [`TracerLocalSparsityDetector`](@ref) are easy to compute using operator overloading by using [dual numbers](@ref SparseConnectivityTracer.Dual) + which contain primal values on which we can evaluate comparisons like `>`: + + ```@repl ctrlflow + using SparseConnectivityTracer + + jacobian_sparsity(f, [2, 1], TracerLocalSparsityDetector()) + + jacobian_sparsity(f, [1, 2], TracerLocalSparsityDetector()) + ``` + + The global sparsity pattern is **impossible** to compute when code branches with an if-else condition, + since we can only ever hit one branch during run-time. + If we made comparisons like `>` return `true` or `false`, we'd get the local patterns `[1 0]` and `[0 1]` respectively. + But SCT's policy is to guarantee conservative sparsity patterns, which means that "false positives" (ones) are acceptable, but "false negatives" (zeros) are not. + In my our opinion, the right thing to do here is to throw an error: + + ```@repl ctrlflow + jacobian_sparsity(f, [1, 2], TracerSparsityDetector()) + ``` + +## Function must be composed of generic Julia functions + +SCT can't trace through non-Julia code. +However, if you know the sparsity pattern of an external, non-Julia function, +you might be able to work around it by adding methods on SCT's tracer types. + +## Function types must be generic + +When computing the sparsity pattern of a function, +it must be written generically enough to accept numbers of type `T<:Real` as (or `AbstractArray{<:Real}`) as inputs. + +!!! details "Example: Overly restrictive type annotations" + Let's see this mistake in action: + + ```@example notgeneric + using SparseConnectivityTracer + method = TracerSparsityDetector() + + relu_bad(x::AbstractFloat) = max(zero(x), x) + outer_function_bad(xs) = sum(relu_bad, xs) + nothing # hide + ``` + + Since tracers and dual numbers are `Real` numbers and not `AbstractFloat`s, + `relu_bad` throws a `MethodError`: + + ```@repl notgeneric + xs = [1.0, -2.0, 3.0]; + + outer_function_bad(xs) + + jacobian_sparsity(outer_function_bad, xs, method) + ``` + + This is easily fixed by loosening type restrictions or adding an additional methods on `Real`: + + ```@example notgeneric + relu_good(x) = max(zero(x), x) + outer_function_good(xs) = sum(relu_good, xs) + nothing # hide + ``` + + ```@repl notgeneric + jacobian_sparsity(outer_function_good, xs, method) + ``` From e09c3c2446118746e3864190d1ef1fe0242d98b0 Mon Sep 17 00:00:00 2001 From: adrhill Date: Tue, 20 Aug 2024 18:00:45 +0200 Subject: [PATCH 12/13] Document `ifelse` --- docs/src/user/api.md | 1 - docs/src/user/limitations.md | 20 ++++++++++++++------ 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/docs/src/user/api.md b/docs/src/user/api.md index 3753393d..a9d84470 100644 --- a/docs/src/user/api.md +++ b/docs/src/user/api.md @@ -7,7 +7,6 @@ CollapsedDocStrings = true # [API Reference](@id api) SparseConnectivityTracer uses [ADTypes.jl](https://github.com/SciML/ADTypes.jl)'s [interface for sparsity detection](https://sciml.github.io/ADTypes.jl/stable/#Sparsity-detector). - In fact, the functions `jacobian_sparsity` and `hessian_sparsity` are re-exported from ADTypes. To compute **global** sparsity patterns of `f(x)` over the entire input domain `x`, use diff --git a/docs/src/user/limitations.md b/docs/src/user/limitations.md index 60a363e9..6db3fd18 100644 --- a/docs/src/user/limitations.md +++ b/docs/src/user/limitations.md @@ -10,14 +10,13 @@ If you observe an overly conservative pattern, [please open a feature request](h ## Limited control flow -[`TracerSparsityDetector`](@ref) does not support any control flow or boolean functions. +Only [`TracerLocalSparsityDetector`](@ref) supports comparison operators (`<`, `==`, ...), indicator functions (`iszero`, `iseven`, ...) and control flow. + +[`TracerSparsityDetector`](@ref) does not support any boolean functions and control flow (with the exception of `iselse`). This might seem unintuitive but follows from our policy stated above: SCT guarantees conservative sparsity patterns. -Using an approach based on operator-overloading, this means that global sparsity detection isn't allowed to hit any branching code. +Using an approach based on operator-overloading, this means that global sparsity detection isn't allowed to hit any branching code. +`ifelse` is the only exception, since it allows us to evaluate both branches. -Only [`TracerLocalSparsityDetector`](@ref) supports limited amounts of comparison operators and control flow, i.e.: -* comparison operators (`<`, `==`, ...) -* indicator functions (`iszero`, `iseven`, ...) -* `ifelse` statements !!! warning "Common control flow errors" By design, SCT will throw errors instead of returning wrong sparsity patterns. Common error messages include: @@ -71,6 +70,15 @@ Only [`TracerLocalSparsityDetector`](@ref) supports limited amounts of compariso jacobian_sparsity(f, [1, 2], TracerSparsityDetector()) ``` + In some cases, we can work around this by using `ifelse`. + Since `ifelse` is a method, we can overload it to evaluate "both branches" and take a conservative union of both resulting sparsity patterns: + + ```@repl ctrlflow + f(x) = ifelse(x[1] > x[2], x[1], x[2]) + + jacobian_sparsity(f, [1, 2], TracerSparsityDetector()) + ``` + ## Function must be composed of generic Julia functions SCT can't trace through non-Julia code. From 14472ffcd05c2bc08eff47b6c7e0e2d1269ccfcd Mon Sep 17 00:00:00 2001 From: adrhill Date: Tue, 20 Aug 2024 18:09:51 +0200 Subject: [PATCH 13/13] Reorder contents --- docs/src/user/limitations.md | 98 ++++++++++++++++++------------------ 1 file changed, 49 insertions(+), 49 deletions(-) diff --git a/docs/src/user/limitations.md b/docs/src/user/limitations.md index 6db3fd18..0ce3662f 100644 --- a/docs/src/user/limitations.md +++ b/docs/src/user/limitations.md @@ -3,11 +3,57 @@ ## Sparsity patterns are conservative approximations Sparsity patterns returned by SparseConnectivityTracer (SCT) can in some cases be overly conservative, meaning that they might contain "too many ones". -If you observe an overly conservative pattern, [please open a feature request](https://github.com/adrhill/SparseConnectivityTracer.jl/issues) so we know where to add more method overloads. +If you observe an overly conservative pattern, [please open a feature request](https://github.com/adrhill/SparseConnectivityTracer.jl/issues) so we know where to add more method overloads to increase the sparsity. -!!! warning "No-false-negatives policy" +!!! warning "SCT's no-false-negatives policy" If you ever observe a sparsity pattern that contains too many zeros, we urge you to [open a bug report](https://github.com/adrhill/SparseConnectivityTracer.jl/issues)! +## Function must be composed of generic Julia functions + +SCT can't trace through non-Julia code. +However, if you know the sparsity pattern of an external, non-Julia function, +you might be able to work around it by adding methods on SCT's tracer types. + +## Function types must be generic + +When computing the sparsity pattern of a function, +it must be written generically enough to accept numbers of type `T<:Real` as (or `AbstractArray{<:Real}`) as inputs. + +!!! details "Example: Overly restrictive type annotations" + Let's see this mistake in action: + + ```@example notgeneric + using SparseConnectivityTracer + method = TracerSparsityDetector() + + relu_bad(x::AbstractFloat) = max(zero(x), x) + outer_function_bad(xs) = sum(relu_bad, xs) + nothing # hide + ``` + + Since tracers and dual numbers are `Real` numbers and not `AbstractFloat`s, + `relu_bad` throws a `MethodError`: + + ```@repl notgeneric + xs = [1.0, -2.0, 3.0]; + + outer_function_bad(xs) + + jacobian_sparsity(outer_function_bad, xs, method) + ``` + + This is easily fixed by loosening type restrictions or adding an additional methods on `Real`: + + ```@example notgeneric + relu_good(x) = max(zero(x), x) + outer_function_good(xs) = sum(relu_good, xs) + nothing # hide + ``` + + ```@repl notgeneric + jacobian_sparsity(outer_function_good, xs, method) + ``` + ## Limited control flow Only [`TracerLocalSparsityDetector`](@ref) supports comparison operators (`<`, `==`, ...), indicator functions (`iszero`, `iseven`, ...) and control flow. @@ -71,56 +117,10 @@ Using an approach based on operator-overloading, this means that global sparsity ``` In some cases, we can work around this by using `ifelse`. - Since `ifelse` is a method, we can overload it to evaluate "both branches" and take a conservative union of both resulting sparsity patterns: + Since `ifelse` is a method, it can evaluate "both branches" and take a conservative union of both resulting sparsity patterns: ```@repl ctrlflow f(x) = ifelse(x[1] > x[2], x[1], x[2]) jacobian_sparsity(f, [1, 2], TracerSparsityDetector()) ``` - -## Function must be composed of generic Julia functions - -SCT can't trace through non-Julia code. -However, if you know the sparsity pattern of an external, non-Julia function, -you might be able to work around it by adding methods on SCT's tracer types. - -## Function types must be generic - -When computing the sparsity pattern of a function, -it must be written generically enough to accept numbers of type `T<:Real` as (or `AbstractArray{<:Real}`) as inputs. - -!!! details "Example: Overly restrictive type annotations" - Let's see this mistake in action: - - ```@example notgeneric - using SparseConnectivityTracer - method = TracerSparsityDetector() - - relu_bad(x::AbstractFloat) = max(zero(x), x) - outer_function_bad(xs) = sum(relu_bad, xs) - nothing # hide - ``` - - Since tracers and dual numbers are `Real` numbers and not `AbstractFloat`s, - `relu_bad` throws a `MethodError`: - - ```@repl notgeneric - xs = [1.0, -2.0, 3.0]; - - outer_function_bad(xs) - - jacobian_sparsity(outer_function_bad, xs, method) - ``` - - This is easily fixed by loosening type restrictions or adding an additional methods on `Real`: - - ```@example notgeneric - relu_good(x) = max(zero(x), x) - outer_function_good(xs) = sum(relu_good, xs) - nothing # hide - ``` - - ```@repl notgeneric - jacobian_sparsity(outer_function_good, xs, method) - ```