Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix floating point number comparison in BatchNorm #14

Closed
wants to merge 275 commits into from
Closed

Fix floating point number comparison in BatchNorm #14

wants to merge 275 commits into from

Conversation

TaoLv
Copy link

@TaoLv TaoLv commented Jan 11, 2018

Description

Fix floating point number comparison in BatchNorm:

  • using fabs(a-b) < 1e-8 for double precision floating point number
  • using fabs(a-b) < 1e-6 for single precision floating point number

Checklist

Essentials

  • Passed code style checking (make lint)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • For user-facing API changes, API doc string has been updated. For new C++ functions in header files, their functionalities and arguments are well-documented.
  • To my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

  • Feature1, tests, (and when applicable, API doc)
  • Feature2, tests, (and when applicable, API doc)

Comments

  • If this change is a backward incompatible change, why must this change be made.
  • Interesting edge cases to note here

@zheng-da
Make dropout stateless.
046eb81
@zheng-da
Make batchnorm stateless.
f4c6f1c
@zheng-da
Make SoftmaxActivation stateless.
30b5fd9
@zheng-da
Fix a code style problem.
95ef90e
@zheng-da
pass amalgamation test for batch norm.
921859a
@zheng-da
pass amalgamation test for dropout.
485f58f
@zheng-da
Get convolution ops from a function.
660968f
@zheng-da
Fix compilation errors for GPU.
26e9430
@zheng-da
Fix thread local in diff platforms.
5504e2c
@zheng-da
Avoid using thread_local for non-CuDNN conv/deconv.
6324176
@zheng-da
Remove TODO in deconv.
36c466f
@zheng-da
Fix a compilation error in dropout.
6410684
@zheng-da
Fix a bug in batch norm.
1fa3898
@zheng-da
Fix a bug in fully connected.
588383a
@zheng-da
Don't set #inputs for backward convolution.
66a281a
@zheng-da
Remove MKL code.
d3ce902
@zheng-da
Update MXNet for MKLDNN.
caa3bf3
@zheng-da
Enable MKLDNN Relu.
db10bb1
@zheng-da
Fix a compilation error.
99c1e08
@zheng-da
Change Makefile for MKLDNN.
a6c2c82
@zheng-da
Remove infer storage in convolution.
3f75f52
@zheng-da
Update MXNet for MKLDNN.
edf6842
@zheng-da
Support MKLDNN storage type in python.
c96ca26
@zheng-da
Update activation.
1a6e06e
@zheng-da
Add MKLDNN base classes.
ca30cac
@zheng-da
Implement MKLDNN fully connected.
79c563c
@zheng-da
Add MKLDNN convolution.
2f5ed28
@zheng-da
Update MKLDNN interface in NDArray.
126b85e
@zheng-da
MKLDNN convolution handle CreateMKLDNNData failure.
7672608
@zheng-da
Add another GetMKLDNNData in NDArray.
4064bef
zheng-da and others added 21 commits January 2, 2018 03:19
@zheng-da
Merge branch 'master' of https://github.com/apache/incubator-mxnet in…
0ff5d85 
…to refactor
@zheng-da
Merge branch 'refactor' of https://github.com/zheng-da/incubator-mxnet …
489603e 
…into refactor
@zheng-da
fix complains from "make lint".
4eeffc9
@zheng-da
Avoid reallocation in NDArray.
f4b73db
@zheng-da
Merge branch 'refactor' of https://github.com/zheng-da/incubator-mxnet …
c8578c0 
…into refactor
@zheng-da
Handle weight arrays with special MKLDNN layouts.
ac8f9fd
@zheng-da
Remove unnecessary GetWeights.
24200a0
@zheng-da
Fix compilation error without MKLDNN.
19d8749
@zheng-da
Fix a bug in (de)conv for weight arrays.
18236fc
@zheng-da
Fix a minor bug in MKLDNN conv.
1cd8bad
@zheng-da
Avoid caching TBlob from NDArray.
9b3c8b2 
This commit may add some overhead of managing NDArray for each fallback.
@zheng-da
Fix a bug in MKLDNNOpSignature.
c426bfa
@TaoLv
Merge remote-tracking branch 'da/refactor' into bn-primitive
623b994 
Conflicts:
	src/operator/nn/mkldnn/mkldnn_batch_norm-inl.h
@TaoLv
1. Fix coding style in BatchNorm;
5825191 
2. Add memory into signature;
3. Try to split BatchNorm into .h file and .cc file. Will finish it after backward code is refactored.
@zheng-da
Merge pull request #5 from TaoLv/bn-primitive
87fd9d5 
Caching primitive for BatchNorm forward computation
@zheng-da
Update mkldnn_base-inl.h
7c957ef
OP primitive cache: use memory as signature for MKLDNN storage type
b4cdd13
@TaoLv
Add primitive caching for Pooling forward computation
15199c7
@zheng-da
Merge pull request zheng-da#13 from TaoLv/pooling-primitive
e814b3e 
Add primitive caching for Pooling forward computation
@zheng-da
Merge pull request zheng-da#11 from jinhuang415/refactor_pr1
1cba97f 
OP primitive cache: use memory as signature for MKLDNN storage type
@TaoLv
Fix floating point number comparison in BatchNorm
adb80dd
@zheng-da
Copy link
Owner

i don't think it's necessary. the parameters are provided by users. if they are different, we can consider them different.

@TaoLv
Copy link
Author

TaoLv commented Jan 11, 2018

If user provides 1/sqrt(2) twice, for user, they are same, but for fp number, there may be a slight difference. Just a concern, you can keep it as is. :)

@zheng-da zheng-da force-pushed the refactor branch 5 times, most recently from d6402e0 to ea46c2f Compare January 15, 2018 05:20
@zheng-da zheng-da closed this Jan 15, 2018
TaoLv pushed a commit to TaoLv/incubator-mxnet that referenced this pull request Aug 8, 2019
@roywei @eric-haibin-lin
[MXNET-1358] Fit api tutorial (apache#15353)
4b07386 
* Added tutorial for FIT API

* Added tests for Fit API tutorial

* Updated index.md for the new tutorial to show up

* Addressed PR feedback

* Addressed PR feedback

* Removed spurious comment for Py2 and Py3 compatibility

* Address PR feedback

* Addressed PR feedback

* Fixed typo

* Added example to showcase custom event handler

* Fixed imports as estimator moved to contrib package

* Added a side note to inform about estimator reference being updated by the handlers

* Corrected typo

* update tutorial

* address comments

* new line

* fix import

* fix cached graph

* fix import

* address comments

* fix doc gen

* add softmax

* add to website index

* fix doc string

* Fix doc gen (zheng-da#12)

* fix warining

* fix test

* fix

* fix

* fix print

* fix test (zheng-da#13)

* fix warning (zheng-da#14)

* fix href (zheng-da#15)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@TaoLv @zheng-da @ashokei @wentingj @pengzhao-intel