Skip to content

[Explainability Evaluation] - Characterization score, and other ways to combine Fid+/- scores #5960

Closed
@BlazStojanovic

Description

@BlazStojanovic

🚀 The feature, motivation and pitch

Implement functionality to combine Fidelity +/- metrics (#5958) features for the purpose of evaluation explainability. This feature request is a part of broader explainability evaluation metrics (parent issue #5628) which will be crucial for the new revamp of explainability (see roadmap #5520).

Brief overview

Fidelity measures check the explanations for their faithfulness to the model. In [1] there are several ways proposed for combining both fidelity scores, like

The area under the $\mathrm{Fid}+/(1-\text { Fid-) }$ curve - AUC

The characterization score: $\operatorname{charact} = \frac{w_{+}+w_{-}}{\frac{w_{+}}{f i d_{+}}+\frac{w_{-}}{1-f i d_{-}}}$

Implementation checklist

  • Implement the characterization score
  • Implement the AUC

References

  1. GraphFramEx: Towards Systematic Evaluation of Explainability Methods for Graph Neural Networks

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions