Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

using Zygote #669

Merged
merged 92 commits into from
Sep 11, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
aa4d221
break all the things
MikeInnes Mar 8, 2019
c313be8
rm data/param
MikeInnes Mar 8, 2019
82ee61f
implement #643
MikeInnes Mar 8, 2019
f9d8ea8
move jacobian test to Tracker
MikeInnes Mar 8, 2019
0c265f3
fix most tests
MikeInnes Mar 8, 2019
5b79453
passing tests... ish
MikeInnes Mar 8, 2019
3182c1b
test on 1.1
MikeInnes Mar 8, 2019
2566952
rm optimiser deprecations
MikeInnes Mar 12, 2019
2bb0c1e
update stuff
MikeInnes Mar 12, 2019
c70276d
rm more deprecations
MikeInnes Mar 12, 2019
92ddc61
update for arrays
MikeInnes Apr 5, 2019
fecb6bd
Update `Manifest`
staticfloat May 3, 2019
0ddb5f0
Tests for Optimisers supporting Zygote
thebhatman Jun 5, 2019
ef63f80
No ops defined for param and data
thebhatman Jun 10, 2019
a782524
Temporarily removed tests of cudnn and curnn.
thebhatman Jun 10, 2019
94a2d19
Updated tests of normalisation layers.
thebhatman Jun 11, 2019
f465665
Corrected test for asymmetric padding
thebhatman Jun 11, 2019
a56cfb7
BatchNorm test corrected
thebhatman Jun 11, 2019
11073dc
GroupNorm made to use istraining()
thebhatman Jun 11, 2019
dfd2965
GroupNorm tests corrected
thebhatman Jun 11, 2019
c7c0ee2
Resolving Merge Conflicts
thebhatman Jun 12, 2019
bd7e3b1
Dropout with dims test passing.
thebhatman Jun 12, 2019
00a4f4c
Correcting Dropout
thebhatman Jun 12, 2019
e979740
DepthwiseConv corrected again.
thebhatman Jun 12, 2019
48ed93c
Silly error in Dropout corrected.
thebhatman Jun 12, 2019
ce11804
CrossCor test passing, hopefully.
thebhatman Jun 12, 2019
1ff4e31
back on mse failing for Float16
thebhatman Jun 13, 2019
25f74d1
Modified tests in cuda.jl
thebhatman Jun 13, 2019
80c680c
Updated tests in cudnn.jl
thebhatman Jun 13, 2019
ce6a1bf
Modifying tests in curnn.jl
thebhatman Jun 13, 2019
7ab9d8e
Minor update
thebhatman Jun 13, 2019
e6d5846
Temporary removal of Float16 test
thebhatman Jun 14, 2019
b194e7e
Callback being called now
thebhatman Jun 19, 2019
f1bf399
nograd defined for sleep
thebhatman Jun 19, 2019
618f8a0
Hopefully the tests pass
thebhatman Jun 19, 2019
9f6793d
Project.toml and Manifest updated
thebhatman Jul 2, 2019
517219b
Renamed gradients test file
thebhatman Jul 2, 2019
3ee2a76
Removed .data from LSTMCell
thebhatman Jul 2, 2019
4e9f3de
Manifest updated with new Zygote version
thebhatman Jul 2, 2019
8292cfd
Decay checking test added back
thebhatman Jul 2, 2019
812541f
zeros replaced by fill to avoid nothing grad
thebhatman Jul 6, 2019
8d78b43
Merge branch 'sf/zygote_updated' of https://github.com/thebhatman/Flu…
thebhatman Jul 8, 2019
cf5bc80
Check for nothing in update step
thebhatman Jul 8, 2019
c2cd7da
re-export gradient
MikeInnes Jul 11, 2019
11c9a84
Remove active from GroupNorm
thebhatman Jul 11, 2019
33c8d84
cuparam -> cuarray
MikeInnes Jul 11, 2019
2b379d0
Allow scalar indexing or onehotbatch tests will fail
thebhatman Jul 12, 2019
c9663c1
pkg up
MikeInnes Jul 12, 2019
e2bf46b
gpu test fixes
MikeInnes Jul 12, 2019
c9cb729
rm REQUIRE
MikeInnes Jul 12, 2019
094b38a
require julia 1.1
MikeInnes Jul 12, 2019
1fc5841
fix dropout
MikeInnes Jul 12, 2019
a140c31
fix batchnorm
MikeInnes Jul 12, 2019
8d6028e
tests with gradients
thebhatman Jul 12, 2019
fc1c0d5
Merge branch 'zygote' of https://github.com/FluxML/Flux.jl into zygote
thebhatman Jul 12, 2019
4ef5ec0
brackets corrected
thebhatman Jul 12, 2019
2816fbb
Fix for getindex error in BatchNorm
thebhatman Jul 12, 2019
d0b94b8
Merge branch 'zygote' of https://github.com/FluxML/Flux.jl into zygote
thebhatman Jul 12, 2019
a128a77
gradients test updated in cudnn
thebhatman Jul 16, 2019
b779d43
replaced trunc Int with div
thebhatman Jul 16, 2019
a645a86
Manifest updated
thebhatman Jul 17, 2019
faac0ff
Updated InstanceNorm and GroupNorm to avoid mutation
thebhatman Jul 18, 2019
f3551da
dropout printing
MikeInnes Jul 24, 2019
b8fabad
deprecate param/data
MikeInnes Aug 19, 2019
49044df
avoid adjoint on abstract type
MikeInnes Aug 19, 2019
3ecca43
formatting fix
MikeInnes Aug 19, 2019
8456b7b
Remove param from groupnorm
thebhatman Aug 19, 2019
a76e4d1
Remove param from crosscor
thebhatman Aug 19, 2019
9590aa6
rm last uses of param/data
MikeInnes Aug 19, 2019
2f7ad89
test cleanups
MikeInnes Aug 19, 2019
447fd9d
conv docstring formatting
MikeInnes Aug 19, 2019
6c67404
update cleanup
MikeInnes Aug 19, 2019
62ec01a
doc build changes
MikeInnes Aug 19, 2019
487000a
fix cuda code and tests
MikeInnes Aug 19, 2019
ee74f1a
pkg up
MikeInnes Aug 22, 2019
2f1a187
Update AlphaDropout
thebhatman Aug 30, 2019
c3cc4bf
Remove double docstring
thebhatman Aug 30, 2019
4ca3204
pkg up
MikeInnes Sep 6, 2019
ecc9ce9
Gradient on AlphaDropout now working
thebhatman Sep 6, 2019
67c38b3
Merge branch 'master' into zygote
MikeInnes Sep 6, 2019
c8d460f
doctests passing
MikeInnes Sep 10, 2019
ddf06af
remove tracker docs
MikeInnes Sep 10, 2019
de20494
docs mostly fixed
MikeInnes Sep 10, 2019
221313c
formatting changed on 1.1
MikeInnes Sep 10, 2019
877415b
rm gradient checks
MikeInnes Sep 10, 2019
b6c8312
optimiser docs
Sep 10, 2019
250aef5
normalise test fixes
MikeInnes Sep 10, 2019
a9d1cbf
added decays
Sep 10, 2019
6fd66fd
Merge branch 'zygote' of https://github.com/FluxML/Flux.jl into zygote
Sep 10, 2019
b08c949
fixes to saving
Sep 11, 2019
b6926f0
cleanup
Sep 11, 2019
e027613
Update docs/src/training/optimisers.md
DhairyaLGandhi Sep 11, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ os:
# - osx

julia:
- 1.0
- 1.1
- nightly

matrix:
Expand Down
41 changes: 27 additions & 14 deletions Manifest.toml
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,12 @@ git-tree-sha1 = "dd169c636d1d3656a9faca772f5bd7c226a61254"
uuid = "0c68f7d7-f131-5f86-a1c3-88cf8149b2d7"
version = "1.0.1"

[[IRTools]]
deps = ["InteractiveUtils", "MacroTools", "Test"]
git-tree-sha1 = "e23faa71b8f54c3fdc99b230b9c2906cafdddca5"
uuid = "7869d1d1-7146-5819-86e3-90919afe41df"
version = "0.2.3"

[[InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
Expand Down Expand Up @@ -226,10 +232,9 @@ uuid = "e89f7d12-3494-54d1-8411-f7d8b9ae1f27"
version = "0.5.0"

[[Missings]]
deps = ["SparseArrays", "Test"]
git-tree-sha1 = "f0719736664b4358aa9ec173077d4285775f8007"
git-tree-sha1 = "29858ce6c8ae629cf2d733bffa329619a1c843d0"
uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
version = "0.4.1"
version = "0.4.2"

[[Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"
Expand All @@ -254,9 +259,9 @@ version = "1.1.0"

[[Parsers]]
deps = ["Dates", "Test"]
git-tree-sha1 = "db2b35dedab3c0e46dc15996d170af07a5ab91c9"
git-tree-sha1 = "ef0af6c8601db18c282d092ccbd2f01f3f0cd70b"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "0.3.6"
version = "0.3.7"

[[Pkg]]
deps = ["Dates", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "UUIDs"]
Expand Down Expand Up @@ -314,10 +319,10 @@ deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[SpecialFunctions]]
deps = ["BinDeps", "BinaryProvider", "Libdl", "Test"]
git-tree-sha1 = "0b45dc2e45ed77f445617b99ff2adf0f5b0f23ea"
deps = ["BinDeps", "BinaryProvider", "Libdl"]
git-tree-sha1 = "3bdd374b6fd78faf0119b8c5d538788dbf910c6e"
uuid = "276daf66-3868-5448-9aa4-cd146d93841b"
version = "0.7.2"
version = "0.8.0"

[[StaticArrays]]
deps = ["LinearAlgebra", "Random", "Statistics"]
Expand Down Expand Up @@ -350,12 +355,6 @@ git-tree-sha1 = "dfcdbbfb2d0370716c815cbd6f8a364efb6f42cf"
uuid = "0796e94c-ce3b-5d07-9a54-7f471281c624"
version = "0.5.6"

[[Tracker]]
deps = ["Adapt", "DiffRules", "ForwardDiff", "LinearAlgebra", "MacroTools", "NNlib", "NaNMath", "Printf", "Random", "Requires", "SpecialFunctions", "Statistics", "Test"]
git-tree-sha1 = "1aa443d3b4bfa91a8aec32f169a479cb87309910"
uuid = "9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c"
version = "0.2.3"

[[TranscodingStreams]]
deps = ["Random", "Test"]
git-tree-sha1 = "7c53c35547de1c5b9d46a4797cf6d8253807108c"
Expand Down Expand Up @@ -386,3 +385,17 @@ deps = ["BinaryProvider", "Libdl", "Printf"]
git-tree-sha1 = "580ce62b6c14244916cc28ad54f8a2e2886f843d"
uuid = "a5390f91-8eb1-5f08-bee0-b1d1ffed6cea"
version = "0.8.3"

[[Zygote]]
deps = ["DiffRules", "FFTW", "FillArrays", "ForwardDiff", "IRTools", "InteractiveUtils", "LinearAlgebra", "MacroTools", "NNlib", "NaNMath", "Random", "Requires", "SpecialFunctions", "Statistics", "ZygoteRules"]
git-tree-sha1 = "9186cb0b3b59219e4aba0840614d6a9d7282012e"
repo-rev = "master"
repo-url = "https://github.com/FluxML/Zygote.jl.git"
uuid = "e88e6eb3-aa80-5325-afca-941959d7151f"
version = "0.3.4"

[[ZygoteRules]]
deps = ["MacroTools"]
git-tree-sha1 = "def5f96ac2895fd9b48435f6b97020979ee0a4c6"
uuid = "700de1a5-db45-46bc-99cf-38207098b444"
version = "0.1.0"
10 changes: 6 additions & 4 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,18 +21,20 @@ Reexport = "189a3867-3050-52da-a836-e630ba90ab69"
SHA = "ea8e919c-243c-51af-8825-aaa63cd721ce"
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
Tracker = "9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
ZipFile = "a5390f91-8eb1-5f08-bee0-b1d1ffed6cea"
Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f"

[compat]
CUDAapi = "1.1"
CuArrays = "1.2"
NNlib = "0.6"
Tracker = "0.2"
julia = "0.7, 1"
Zygote = "0.3"
julia = "1.1"

[extras]
Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[targets]
test = ["Test"]
test = ["Test", "Documenter"]
13 changes: 0 additions & 13 deletions REQUIRE

This file was deleted.

Loading