You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Implement functionality to combine Fidelity +/- metrics (#5958) features for the purpose of evaluation explainability. This feature request is a part of broader explainability evaluation metrics (parent issue #5628) which will be crucial for the new revamp of explainability (see roadmap #5520).
Brief overview
Fidelity measures check the explanations for their faithfulness to the model. In [1] there are several ways proposed for combining both fidelity scores, like
The area under the$\mathrm{Fid}+/(1-\text { Fid-) }$curve - AUC
The characterization score: $\operatorname{charact} = \frac{w_{+}+w_{-}}{\frac{w_{+}}{f i d_{+}}+\frac{w_{-}}{1-f i d_{-}}}$
…to combine Fid+/- scores (#6188)
This PR implements the features from #5960
The tests and the math part is done, I'm somewhat lost with the sphinx
documentation and very much open to suggestions for the docstrings.
As #5962 also requires calculating an AUC and probably a weighted
harmonic mean for F_1 type scores these parts have been implemented as
separate helper functions.
Co-authored-by: Blaž Stojanovič <Blaz@stojanovic.si>
Co-authored-by: rusty1s <matthias.fey@tu-dortmund.de>
🚀 The feature, motivation and pitch
Implement functionality to combine Fidelity +/- metrics (#5958) features for the purpose of evaluation explainability. This feature request is a part of broader explainability evaluation metrics (parent issue #5628) which will be crucial for the new revamp of explainability (see roadmap #5520).
Brief overview
Fidelity measures check the explanations for their faithfulness to the model. In [1] there are several ways proposed for combining both fidelity scores, like
The area under the$\mathrm{Fid}+/(1-\text { Fid-) }$ curve - AUC
The characterization score:$\operatorname{charact} = \frac{w_{+}+w_{-}}{\frac{w_{+}}{f i d_{+}}+\frac{w_{-}}{1-f i d_{-}}}$
Implementation checklist
References
The text was updated successfully, but these errors were encountered: