Skip to content

doctest update metrics #2144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

nataliyah123
Copy link
Contributor

@nataliyah123 nataliyah123 commented Sep 2, 2020

Fixes part of # 2066

@bot-of-gabrieldemarmiesse

@AakashKumarNain @marload

You are owners of some files modified in this pull request.
Would you kindly review the changes whenever you have the time to?
Thank you very much.

@nataliyah123 nataliyah123 changed the title update metrics cohen kappa [WIP]update metrics cohen kappa Sep 2, 2020
@nataliyah123 nataliyah123 changed the title [WIP]update metrics cohen kappa [WIP] doctest update metrics Sep 2, 2020
@nataliyah123 nataliyah123 requested a review from autoih as a code owner September 2, 2020 16:40
nataliyah123 and others added 18 commits September 2, 2020 22:23
* updated contrastive.py

* updated focal_loss.py

* updated giou_loss.py

* updated kappa_loss.py

* updated npairs.py

* updated quantiles.py

* reformatting giou_loss.py

* updated testable docs contrastive.py

* updated doctests focal_loss.py

* updated testdocs giou_loss.py

* updated testdocs kappa_loss.py

* updated testdocs npairs.py

* updated testdocs quantiles.py

* minor changes to formatting

* fixing multi-line error

* adding empty line inbetween imports

* adding empty line inbetween imports

* reformatting imports

* reverting code formatting changes

* reformatting imports
@nataliyah123 nataliyah123 changed the title [WIP] doctest update metrics doctest update metrics Sep 5, 2020
@WindQAQ WindQAQ self-requested a review September 5, 2020 18:58
Copy link
Member

@WindQAQ WindQAQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! Can we canonicalize the variable names? Like

  • actuals -> y_true
  • preds -> y_pred
  • all metric instance should be named metric
  • all results from metric(y_true, y_pred) should be named result.

>>> m = tfa.metrics.CohenKappa(num_classes=5, sparse_labels=True)
>>> # m.update_state(actuals, preds)
>>> # print('Final result: ', m.result().numpy())
>>> # Final result: 0.61904764
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why these are commented out?

Copy link
Contributor Author

@nataliyah123 nataliyah123 Sep 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please see the image below.
I don't reckon what do you mean by 'all results from metric(y_true, y_pred) should be named result'. do you mean whenever a metric is directly called or do you mean metric.update_state(y,ypred), should be assigned to result?. if I assign update_state to result the print statement should be like
print('Final result: ', result.result().numpy())
can you please check this PR. it has a similar issue as described below. I have not reverted back to the state where all checks were cleared.
image

>>> m = tfa.metrics.CohenKappa(num_classes=5, sparse_labels=True)
>>> # m.update_state(actuals, preds, sample_weight=weights)
>>> # print('Final result: ', m.result().numpy())
>>> # Final result: 0.37209308
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not complete ditto the second example uses weights.

>>> preds = tf.constant([[1.0], [0.0], [1.0], [1.0]],
... dtype=tf.float32)
>>> # Matthews correlation coefficient
>>> mcc = MatthewsCorrelationCoefficient(num_classes=1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tfa.metrics.MatthewsCorrelationCoefficient?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had my doubt about this but since it was working completely fine. So, I left it as it is.

@WindQAQ
Copy link
Member

WindQAQ commented Sep 22, 2020

This is probably stale as #2171 exists. Let me close it but feel free to reopen it if I make the mistake.

@WindQAQ WindQAQ closed this Sep 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.