Adding robustness metrics for binary classification #167

fracarfer5 · 2024-07-22T17:47:01Z

Adding source code for empirical robustness and adversarial accuracy metrics
Adding source code for binary classification attackers (HSJ and Zoo)
Adding unit tests
Adding documentation
Adding implementation tutorial

* many things * merge struct and docs

* docs: formatting docstrings to numpy-like docs format * chore: updating pyproject to ignore mitigation folder * refactor: updating code with recommended ruff practices * fix: updating pyproject for ruff support * chore: updating pyprojectt * chore: updating pyproject

* docs: updating docstrings for commons in bias mitigation methods * docs: updating docstrings for preprocessing bias mitigation methods

* many things * merge struct and docs * update dataset and mitigation

* update dataet map function * filter * ok * fix: headers

* feat: adding seed to allow reproducibility to synthetic generation ranking * feat: adding recsys dataset and tolerance threshold parameter in check results function * tests: adding tests for recsys mitigators for in-, pre-, and post-processing methods * refactor: refactoring recsys tutorials with new dataset function * refactor: refactoring recsys measuring bias tutorial with new dataset and updated paths

Co-authored-by: crismunoz <cristian.munoz@holisticai.com>

* reafactory: clean files * fix: ruff * update ruff

* fix: dataset and test * fix: format * chore: fix name * chore: add cache dir * chore: create cache dir * chore: upate python viersion * update with new versions numpy * Update workflow * chore: udpate types * chore: update dependencies and remove invalid dependencies * chore: update test * chore: update changes * chore: remove import

* docs: adding xai metrics documentation * chore: hatch fmt

…ification tasks

…cation

* chore: update security and exaplainability * test: fix atol

* feat: adding template for class docstrings rendering * docs: updating docstrings for bias mitigation methods * fix: renaming method * fix: disabling ruff for privated method * fix: updating variable name * fix: updating variable name * fix: making sensgroups variable public in reweighing * fix: making sensgroups variable public in reweighing * docs: fixing lint * docs: updating methods and references from docstrings

…o feature/robustness-metrics-binary-classification

* docs: improve docstring and remove old dependenceis * chore: hatch format

* docs: adding descriptions for inprocessing methods * docs: updating method descriptions, adding index and reformatting sectinos * docs: adding learning task note for all methods * docs: adding documentation for adversarial debiasing and debiasing learning methods --------- Co-authored-by: Cristian Muñoz <cristian.munoz@holisticai.com>

…o feature/robustness-metrics-binary-classification

* docs: improve docstring and remove old dependenceis * chore: hatch format * chore: update security info

* chore: partial dependence formatting * chore: fmt

…o feature/robustness-metrics-binary-classification

…/robustness-metrics-binary-classification

crismunoz · 2024-08-02T14:33:01Z

Move your changes to the new branch from main
Remove BinclasAttacker
Add Attacker Tests (sugin 2 examples small parameters) test with 2 cases DataFrame input and Numpy Input.

$fracarfer5$

crismunoz and others added 30 commits May 22, 2024 13:53

v1-draft

a928fe9

update docs folder (#118)

f460e98

new Holistic AI struct (#116)

2d65ab8

* many things * merge struct and docs

fix: plot docs and cookbook for plots (#120)

14e09d2

$@fracarfer5$

docs: updating docstrings for commons in bias mitigation methods (#123)

fa8d211

* docs: updating docstrings for commons in bias mitigation methods * docs: updating docstrings for preprocessing bias mitigation methods

update: inprocessing mitigation docstrings (#124)

182b7e6

Holisticai v1 core (#125)

d5f8de2

* many things * merge struct and docs * update dataset and mitigation

update unused docs

7d54eab

update dataet map function (#127)

6fc96b9

* update dataet map function * filter * ok * fix: headers

tests: bias mitigation for clustering and clustering tutorials (#129)

057a14f

Co-authored-by: crismunoz <cristian.munoz@holisticai.com>

$@fracarfer5$

refactor: updating tutorials with new dataset and updating paths (#132)

d28f6ed

reafactory: clean files (#133)

67b3647

* reafactory: clean files * fix: ruff * update ruff

docs: classification and multiclassification tutorials (#134)

d2961a7

adding logo hai

6e0335b

chore: updat tutorials

d57fdd8

change bias <-> mitigation, metrics

4170d47

chore: fix dependencies

2fb00a7

chore update docs

0dd39e8

fix: build documentation

384b574

chore: fix docs

1a66bbb

chore: readthedocs

e2fb261

chore: fix readthedoscs

ef06813

update tutorials

fd7a945

fix: treat empty oop_metrics

e34445e

chore: update numpy version

936b843

chore: add holisticai.bias.plots

a543262

chore: avoid crash old codes

c854d18

crismunoz and others added 5 commits July 15, 2024 21:25

Feature/documentation xai metrics (#164)

05a199d

* docs: adding xai metrics documentation * chore: hatch fmt

docs: fix images folder static (#165)

53865d6

$@fracarfer5$

feat: adding robustness metrics for binary classification models

e59c8f6

$@fracarfer5$

tests: adding testing modules for robustness metrics for binary class…

4e3a151

…ification tasks

$@fracarfer5$

docs: adding documentation for robustness metrics for binary classifi…

0dc6ede

…cation

$@fracarfer5$ fracarfer5 requested review from crismunoz and Kleyt0n July 22, 2024 17:47

crismunoz and others added 19 commits July 23, 2024 16:44

chore: update security and exaplainability (#168)

42fb589

* chore: update security and exaplainability * test: fix atol

$@fracarfer5$

Merge branch 'holisticai-v1' of github.com:holistic-ai/holisticai int…

e31c0cf

…o feature/robustness-metrics-binary-classification

docs: improve docstring and remove old dependenceis (#169)

e2f388e

* docs: improve docstring and remove old dependenceis * chore: hatch format

$@fracarfer5$

feat: combining binary classification robustness metrics in one file

847959a

$@fracarfer5$

Merge branch 'holisticai-v1' of github.com:holistic-ai/holisticai int…

4bf6c6d

…o feature/robustness-metrics-binary-classification

Feature/rst security (#170)

cfb7440

* docs: improve docstring and remove old dependenceis * chore: hatch format * chore: update security info

docs: update foldername

5132823

chore: fix-format (#171)

43821a2

chore: partial dependence formatting (#172)

d9f2337

* chore: partial dependence formatting * chore: fmt

chore: fix plots generation docs (#173)

43e1457

chore: update install intruction (#174)

bf178ec

$@fracarfer5$

Merge branch 'holisticai-v1' of github.com:holistic-ai/holisticai int…

3327cc9

…o feature/robustness-metrics-binary-classification

$@fracarfer5$

Merge branch 'main' of github.com:holistic-ai/holisticai into feature…

dab6e35

…/robustness-metrics-binary-classification

$@fracarfer5$

feat: adding attackers for robustness module

31a547a

$@fracarfer5$

feat: adding tutorial for robustness module

d5e46e9

$@fracarfer5$

docs: adding and updating docs for robustness attackers

4526a9d

$@fracarfer5$

chore: fixing lint and style

9f172a2

$@fracarfer5$ fracarfer5 changed the base branch from holisticai-v1 to main July 30, 2024 14:45

$@fracarfer5$ fracarfer5 closed this Aug 8, 2024

crismunoz deleted the feature/robustness-metrics-binary-classification branch September 23, 2024 15:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding robustness metrics for binary classification #167

Adding robustness metrics for binary classification #167

$@fracarfer5$ fracarfer5 commented Jul 22, 2024 •

edited

Loading

crismunoz commented Aug 2, 2024

Adding robustness metrics for binary classification #167

Adding robustness metrics for binary classification #167

Conversation

fracarfer5 commented Jul 22, 2024 • edited Loading

crismunoz commented Aug 2, 2024

$@fracarfer5$ fracarfer5 commented Jul 22, 2024 •

edited

Loading