Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add sizes for input layer to support caffe to specify input dimensions #2163

Closed
wants to merge 4 commits into from
Closed

Add sizes for input layer to support caffe to specify input dimensions #2163

wants to merge 4 commits into from

Conversation

wzhongyuan
Copy link
Contributor

What changes were proposed in this pull request?

Add sizes for input layer to support caffe to specify input dimensions

How was this patch tested?

unit test

Related links or issues (optional)

@wzhongyuan wzhongyuan requested a review from yiheng January 8, 2018 01:43
yiheng
yiheng reviewed Jan 8, 2018
View reviewed changes
spark/dl/src/main/scala/com/intel/analytics/bigdl/nn/Input.scala Outdated
@@ -36,7 +36,7 @@ import scala.reflect.ClassTag
* @tparam T The numeric type in the criterion, usually which are [[Float]] or [[Double]]
*/
@SerialVersionUID(- 8525406230282608924L)
class Input[T: ClassTag]()(implicit ev: TensorNumeric[T])
class Input[T: ClassTag](val sizes: Array[Int] = null)(implicit ev: TensorNumeric[T])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest defining a new Shape class. We will use it in shape infer

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we deal with unfixed size, e.g. batch size dimension

yiheng
yiheng reviewed Jan 8, 2018
View reviewed changes
Copy link
Contributor

@yiheng yiheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to add new unit tests for the changes.

@yiheng
Copy link
Contributor

yiheng commented Jan 23, 2018

There's a new shape class has been added in another PR. Maybe we can close this one.

@wzhongyuan
Copy link
Contributor Author

@yiheng yes, but we still need this PR , i will rebase and update

@wzhongyuan
Copy link
Contributor Author

@yiheng rebased and updated, and refined the test files to apply the change
PR validation passed http://172.168.2.101:8080/job/BigDL-PR-Validation-v3/90/

@wzhongyuan wzhongyuan closed this May 23, 2018
Le-Zheng added a commit to Le-Zheng/BigDL that referenced this pull request Oct 20, 2021
@Le-Zheng
run examples on k8s readme (intel-analytics#2163)
79fdfa0 
* k8s  readme
dding3 pushed a commit to dding3/BigDL that referenced this pull request Nov 17, 2021
@glorysdj @Le-Zheng
@pinggao18 @pinggao187 @hkvision @GavinGu07 @fanzhuyifan @Litchilitchy @ardaci @yangw1234 @zzti-bsj @Adria777
Migrate hyperzoo (intel-analytics#4958)
77da6ed 
* add hyperzoo for k8s support (intel-analytics#2140)

* add hyperzoo for k8s support

* format

* format

* format

* format

* run examples on k8s readme (intel-analytics#2163)

* k8s  readme

* fix jdk download issue (intel-analytics#2219)

* add doc for submit jupyter notebook and cluster serving to k8s (intel-analytics#2221)

* add hyperzoo doc

* add hyperzoo doc

* add hyperzoo doc

* add hyperzoo doc

* fix jdk download issue (intel-analytics#2223)

* bump to 0.9s (intel-analytics#2227)

* update jdk download url (intel-analytics#2259)

* update some previous docs (intel-analytics#2284)

* K8docsupdate (intel-analytics#2306)

* Update README.md

* Update s3 related links in readme  and documents (intel-analytics#2489)

* Update s3 related links in readme  and documents

* Update s3 related links in readme and documents

* Update s3 related links in readme and documents

* Update s3 related links in readme and documents

* Update s3 related links in readme and documents

* Update s3 related links in readme and documents

* update

* update

* modify line length limit

* update

* Update mxnet-mkl version in hyper-zoo dockerfile (intel-analytics#2720)

Co-authored-by: gaoping <pingx.gao@intel.com>

* update bigdl version (intel-analytics#2743)

* update bigdl version

* hyperzoo dockerfile add cluster-serving (intel-analytics#2731)

* hyperzoo dockerfile add cluster-serving

* update

* update

* update

* update jdk url

* update jdk url

* update

Co-authored-by: gaoping <pingx.gao@intel.com>

* Support init_spark_on_k8s (intel-analytics#2813)

* initial

* fix

* code refactor

* bug fix

* update docker

* style

* add conda to docker image (intel-analytics#2894)

* add conda to docker image

* Update Dockerfile

* Update Dockerfile

Co-authored-by: glorysdj <glorysdj@gmail.com>

* Fix code blocks indents in .md files (intel-analytics#2978)

* Fix code blocks indents in .md files

Previously a lot of the code blocks in markdown files were horribly indented with bad white spaces in the beginning of lines. Users can't just select, copy, paste, and run (in the case of python). I have fixed all these, so there is no longer any code block with bad white space at the beginning of the lines.
It would be nice if you could try to make sure in future commits that all code blocks are properly indented inside and have the right amount of white space in the beginning!

* Fix small style issue

* Fix indents

* Fix indent and add \ for multiline commands

Change indent from 3 spaces to 4, and add "\" for multiline bash commands

Co-authored-by: Yifan Zhu <fanzhuyifan@gmail.com>

* enable bigdl 0.12 (intel-analytics#3101)

* switch to bigdl 0.12

* Hyperzoo example ref (intel-analytics#3143)

* specify pip version to fix oserror 0 of proxy (intel-analytics#3165)

* Bigdl0.12.1 (intel-analytics#3155)

* bigdl 0.12.1

* bump 0.10.0-Snapshot (intel-analytics#3237)

* update runtime image name (intel-analytics#3250)

* update jdk download url (intel-analytics#3316)

* update jdk8 url (intel-analytics#3411)

Co-authored-by: ardaci <dongjie.shi@intel.com>

* update hyperzoo docker image (intel-analytics#3429)

* update hyperzoo image (intel-analytics#3457)

* fix jdk in az docker (intel-analytics#3478)

* fix jdk in az docker

* fix jdk for hyperzoo

* fix jdk in jenkins docker

* fix jdk in cluster serving docker

* fix jdk

* fix readme

* update python dep to fit cnvrg (intel-analytics#3486)

* update ray version doc (intel-analytics#3568)

* fix deploy hyperzoo issue (intel-analytics#3574)

Co-authored-by: gaoping <pingx.gao@intel.com>

* add spark fix and net-tools and status check (intel-analytics#3742)

* intsall netstat and add check status

* add spark fix for graphene

* bigdl 0.12.2 (intel-analytics#3780)

* bump to 0.11-S and fix version issues except ipynb

* add multi-stage build Dockerfile (intel-analytics#3916)

* add multi-stage build Dockerfile

* multi-stage build dockerfile

* multi-stage build dockerfile

* Rename Dockerfile.multi to Dockerfile

* delete Dockerfile.multi

* remove comments, add TINI_VERSION to common arg, remove Dockerfile.multi

* multi-stage add tf_slim

Co-authored-by: shaojie <shaojiex.bai@intel.com>

* update hyperzoo doc and k8s doc (intel-analytics#3959)

* update userguide of k8s

* update k8s guide

* update hyperzoo doc

* Update k8s.md

add note

* Update k8s.md

add note

* Update k8s.md

update notes

* fix 4087 issue (intel-analytics#4097)

Co-authored-by: shaojie <shaojiex.bai@intel.com>

* fixed 4086 and 4083 issues (intel-analytics#4098)

Co-authored-by: shaojie <shaojiex.bai@intel.com>

* Reduce image size (intel-analytics#4132)

* Reduce Dockerfile size
1. del redis stage
2. del flink stage
3. del conda & exclude some python packages
4. add copies layer stage

* update numpy version to 1.18.1

Co-authored-by: zzti-bsj <shaojiex.bai@intel.com>

* update hyperzoo image (intel-analytics#4250)

Co-authored-by: Adria777 <Adria777@github.com>

* bigdl 0.13 (intel-analytics#4210)

* bigdl 0.13

* update

* print exception

* pyspark2.4.6

* update release PyPI script

* update

* flip snapshot-0.12.0 and spark2.4.6 (intel-analytics#4254)

* s-0.12.0 master

* Update __init__.py

* Update python.md

* fix docker issues due to version update (intel-analytics#4280)

* fix docker issues

* fix docker issues

* update Dockerfile to support spark 3.1.2 && 2.4.6 (intel-analytics#4436)

Co-authored-by: shaojie <otnw_bsj@163.com>

* update hyperzoo, add lib for tf2 (intel-analytics#4614)

* delete tf 1.15.0 (intel-analytics#4719)

Co-authored-by: Le-Zheng <30695225+Le-Zheng@users.noreply.github.com>
Co-authored-by: pinggao18 <44043817+pinggao18@users.noreply.github.com>
Co-authored-by: pinggao187 <44044110+pinggao187@users.noreply.github.com>
Co-authored-by: gaoping <pingx.gao@intel.com>
Co-authored-by: Kai Huang <huangkaivision@gmail.com>
Co-authored-by: GavinGu07 <55721214+GavinGu07@users.noreply.github.com>
Co-authored-by: Yifan Zhu <zhuyifan@stanford.edu>
Co-authored-by: Yifan Zhu <fanzhuyifan@gmail.com>
Co-authored-by: Song Jiaming <litchy233@gmail.com>
Co-authored-by: ardaci <dongjie.shi@intel.com>
Co-authored-by: Yang Wang <yang3.wang@intel.com>
Co-authored-by: zzti-bsj <2779090360@qq.com>
Co-authored-by: shaojie <shaojiex.bai@intel.com>
Co-authored-by: Lingqi Su <33695124+Adria777@users.noreply.github.com>
Co-authored-by: Adria777 <Adria777@github.com>
Co-authored-by: shaojie <otnw_bsj@163.com>
dding3 pushed a commit to dding3/BigDL that referenced this pull request Nov 17, 2021
@glorysdj @Le-Zheng
@pinggao18 @pinggao187 @hkvision @GavinGu07 @fanzhuyifan @Litchilitchy @ardaci @yangw1234 @zzti-bsj @Adria777
Migrate hyperzoo (intel-analytics#4958)
c23a33f 
* add hyperzoo for k8s support (intel-analytics#2140)

* add hyperzoo for k8s support

* format

* format

* format

* format

* run examples on k8s readme (intel-analytics#2163)

* k8s  readme

* fix jdk download issue (intel-analytics#2219)

* add doc for submit jupyter notebook and cluster serving to k8s (intel-analytics#2221)

* add hyperzoo doc

* add hyperzoo doc

* add hyperzoo doc

* add hyperzoo doc

* fix jdk download issue (intel-analytics#2223)

* bump to 0.9s (intel-analytics#2227)

* update jdk download url (intel-analytics#2259)

* update some previous docs (intel-analytics#2284)

* K8docsupdate (intel-analytics#2306)

* Update README.md

* Update s3 related links in readme  and documents (intel-analytics#2489)

* Update s3 related links in readme  and documents

* Update s3 related links in readme and documents

* Update s3 related links in readme and documents

* Update s3 related links in readme and documents

* Update s3 related links in readme and documents

* Update s3 related links in readme and documents

* update

* update

* modify line length limit

* update

* Update mxnet-mkl version in hyper-zoo dockerfile (intel-analytics#2720)

Co-authored-by: gaoping <pingx.gao@intel.com>

* update bigdl version (intel-analytics#2743)

* update bigdl version

* hyperzoo dockerfile add cluster-serving (intel-analytics#2731)

* hyperzoo dockerfile add cluster-serving

* update

* update

* update

* update jdk url

* update jdk url

* update

Co-authored-by: gaoping <pingx.gao@intel.com>

* Support init_spark_on_k8s (intel-analytics#2813)

* initial

* fix

* code refactor

* bug fix

* update docker

* style

* add conda to docker image (intel-analytics#2894)

* add conda to docker image

* Update Dockerfile

* Update Dockerfile

Co-authored-by: glorysdj <glorysdj@gmail.com>

* Fix code blocks indents in .md files (intel-analytics#2978)

* Fix code blocks indents in .md files

Previously a lot of the code blocks in markdown files were horribly indented with bad white spaces in the beginning of lines. Users can't just select, copy, paste, and run (in the case of python). I have fixed all these, so there is no longer any code block with bad white space at the beginning of the lines.
It would be nice if you could try to make sure in future commits that all code blocks are properly indented inside and have the right amount of white space in the beginning!

* Fix small style issue

* Fix indents

* Fix indent and add \ for multiline commands

Change indent from 3 spaces to 4, and add "\" for multiline bash commands

Co-authored-by: Yifan Zhu <fanzhuyifan@gmail.com>

* enable bigdl 0.12 (intel-analytics#3101)

* switch to bigdl 0.12

* Hyperzoo example ref (intel-analytics#3143)

* specify pip version to fix oserror 0 of proxy (intel-analytics#3165)

* Bigdl0.12.1 (intel-analytics#3155)

* bigdl 0.12.1

* bump 0.10.0-Snapshot (intel-analytics#3237)

* update runtime image name (intel-analytics#3250)

* update jdk download url (intel-analytics#3316)

* update jdk8 url (intel-analytics#3411)

Co-authored-by: ardaci <dongjie.shi@intel.com>

* update hyperzoo docker image (intel-analytics#3429)

* update hyperzoo image (intel-analytics#3457)

* fix jdk in az docker (intel-analytics#3478)

* fix jdk in az docker

* fix jdk for hyperzoo

* fix jdk in jenkins docker

* fix jdk in cluster serving docker

* fix jdk

* fix readme

* update python dep to fit cnvrg (intel-analytics#3486)

* update ray version doc (intel-analytics#3568)

* fix deploy hyperzoo issue (intel-analytics#3574)

Co-authored-by: gaoping <pingx.gao@intel.com>

* add spark fix and net-tools and status check (intel-analytics#3742)

* intsall netstat and add check status

* add spark fix for graphene

* bigdl 0.12.2 (intel-analytics#3780)

* bump to 0.11-S and fix version issues except ipynb

* add multi-stage build Dockerfile (intel-analytics#3916)

* add multi-stage build Dockerfile

* multi-stage build dockerfile

* multi-stage build dockerfile

* Rename Dockerfile.multi to Dockerfile

* delete Dockerfile.multi

* remove comments, add TINI_VERSION to common arg, remove Dockerfile.multi

* multi-stage add tf_slim

Co-authored-by: shaojie <shaojiex.bai@intel.com>

* update hyperzoo doc and k8s doc (intel-analytics#3959)

* update userguide of k8s

* update k8s guide

* update hyperzoo doc

* Update k8s.md

add note

* Update k8s.md

add note

* Update k8s.md

update notes

* fix 4087 issue (intel-analytics#4097)

Co-authored-by: shaojie <shaojiex.bai@intel.com>

* fixed 4086 and 4083 issues (intel-analytics#4098)

Co-authored-by: shaojie <shaojiex.bai@intel.com>

* Reduce image size (intel-analytics#4132)

* Reduce Dockerfile size
1. del redis stage
2. del flink stage
3. del conda & exclude some python packages
4. add copies layer stage

* update numpy version to 1.18.1

Co-authored-by: zzti-bsj <shaojiex.bai@intel.com>

* update hyperzoo image (intel-analytics#4250)

Co-authored-by: Adria777 <Adria777@github.com>

* bigdl 0.13 (intel-analytics#4210)

* bigdl 0.13

* update

* print exception

* pyspark2.4.6

* update release PyPI script

* update

* flip snapshot-0.12.0 and spark2.4.6 (intel-analytics#4254)

* s-0.12.0 master

* Update __init__.py

* Update python.md

* fix docker issues due to version update (intel-analytics#4280)

* fix docker issues

* fix docker issues

* update Dockerfile to support spark 3.1.2 && 2.4.6 (intel-analytics#4436)

Co-authored-by: shaojie <otnw_bsj@163.com>

* update hyperzoo, add lib for tf2 (intel-analytics#4614)

* delete tf 1.15.0 (intel-analytics#4719)

Co-authored-by: Le-Zheng <30695225+Le-Zheng@users.noreply.github.com>
Co-authored-by: pinggao18 <44043817+pinggao18@users.noreply.github.com>
Co-authored-by: pinggao187 <44044110+pinggao187@users.noreply.github.com>
Co-authored-by: gaoping <pingx.gao@intel.com>
Co-authored-by: Kai Huang <huangkaivision@gmail.com>
Co-authored-by: GavinGu07 <55721214+GavinGu07@users.noreply.github.com>
Co-authored-by: Yifan Zhu <zhuyifan@stanford.edu>
Co-authored-by: Yifan Zhu <fanzhuyifan@gmail.com>
Co-authored-by: Song Jiaming <litchy233@gmail.com>
Co-authored-by: ardaci <dongjie.shi@intel.com>
Co-authored-by: Yang Wang <yang3.wang@intel.com>
Co-authored-by: zzti-bsj <2779090360@qq.com>
Co-authored-by: shaojie <shaojiex.bai@intel.com>
Co-authored-by: Lingqi Su <33695124+Adria777@users.noreply.github.com>
Co-authored-by: Adria777 <Adria777@github.com>
Co-authored-by: shaojie <otnw_bsj@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yiheng yiheng yiheng left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@wzhongyuan @yiheng