[New feature] Add Boxhead #2894

zhangxiaoli73 · 2019-08-22T01:45:21Z

What changes were proposed in this pull request?

Add box head that is used in MaskRCNN

How was this patch tested?

unit tests

Related links or issues (optional)

fixed https://github.com/intel-analytics/BigDL/issues/XXX

wzhongyuan · 2019-09-05T07:53:22Z

spark/dl/src/main/scala/com/intel/analytics/bigdl/nn/BoxHead.scala

+
+  override def buildModel(): Module[Float] = {
+    val featureExtractor = this.featureExtractor(
+      inChannels, resolution, scales, samplingRatio.toInt, outputSize)


why toInt ? can we just make the type as Int instead of Float ?

ok, good idea. changed

wzhongyuan · 2019-09-05T07:53:38Z

spark/dl/src/main/scala/com/intel/analytics/bigdl/nn/BoxHead.scala

+    val featureExtractor = this.featureExtractor(
+      inChannels, resolution, scales, samplingRatio.toInt, outputSize)
+
+    val clsPre = this.clsPredictor(numClasses, outputSize)


clsPredictor

ok, changed

wzhongyuan · 2019-09-05T07:53:48Z

spark/dl/src/main/scala/com/intel/analytics/bigdl/nn/BoxHead.scala

+      inChannels, resolution, scales, samplingRatio.toInt, outputSize)
+
+    val clsPre = this.clsPredictor(numClasses, outputSize)
+    val bboxPre = this.bboxPredictor(numClasses, outputSize)


bboxPredictor

ok, changed

wzhongyuan · 2019-09-05T07:54:14Z

spark/dl/src/main/scala/com/intel/analytics/bigdl/nn/BoxHead.scala

+    val clsPre = this.clsPredictor(numClasses, outputSize)
+    val bboxPre = this.bboxPredictor(numClasses, outputSize)
+
+    val weight = Array(10.0f, 10.0f, 5.0f, 5.0f)


why weight is hard-coded const value ?

weight is fixed for mask-rcnn model.

wzhongyuan · 2019-09-05T07:55:52Z

spark/dl/src/main/scala/com/intel/analytics/bigdl/nn/BoxHead.scala

+   * applying non-maximum suppression (NMS).
+   */
+  private[nn] def filterResults(boxes: Tensor[Float], scores: Tensor[Float],
+                                num_classes: Int): Array[RoiLabel] = {


numOfClasses

ok, changed

wzhongyuan · 2019-09-05T07:57:36Z

spark/dl/src/main/scala/com/intel/analytics/bigdl/nn/BoxHead.scala

+    BboxUtil.decodeWithWeight(boxRegression, bbox, weight, boxesBuf)
+
+    val boxesInImage = bbox.size(1)
+    val proposals_split = boxesBuf.split(boxesInImage, dim = 1)


proposalSplit

ok, changed

wzhongyuan · 2019-09-05T07:57:44Z

spark/dl/src/main/scala/com/intel/analytics/bigdl/nn/BoxHead.scala

+    if (boxesBuf == null) boxesBuf = Tensor[Float]
+    boxesBuf.resizeAs(boxRegression)
+
+    val class_prob = softMax.forward(classLogits)


ok, changed

wzhongyuan · 2019-09-05T07:58:02Z

spark/dl/src/main/scala/com/intel/analytics/bigdl/nn/BoxHead.scala

+
+    val boxesInImage = bbox.size(1)
+    val proposals_split = boxesBuf.split(boxesInImage, dim = 1)
+    val class_prob_split = class_prob.split(boxesInImage, dim = 1)


classProbSplit

ok, changed

wzhongyuan · 2019-09-05T07:59:22Z

Please change the c++ style naming

i8run · 2019-09-06T09:18:40Z

spark/dl/src/main/scala/com/intel/analytics/bigdl/nn/BoxHead.scala

+
+  private val softMax = SoftMax[Float]()
+  private val nmsTool: Nms = new Nms
+  @transient var boxesBuf: Tensor[Float] = _


make it private.

ok, changed

wzhongyuan · 2019-09-09T02:01:31Z

spark/dl/src/main/scala/com/intel/analytics/bigdl/nn/BoxHead.scala

+    val results = new Array[RoiLabel](numOfClasses)
+    var clsInd = 1
+    while (clsInd < numOfClasses) {
+      results(clsInd) = postProcessOneClass(scores, boxes, clsInd)


where is results[0] ?

class number 0 is background. So it is ignored. Add some comments for it.

wzhongyuan · 2019-09-09T02:04:58Z

spark/dl/src/main/scala/com/intel/analytics/bigdl/nn/BoxHead.scala

+
+  private def selectTensor(matrix: Tensor[Float], indices: Array[Int],
+    dim: Int, indiceLen: Int = -1, out: Tensor[Float] = null): Tensor[Float] = {
+    assert(dim == 1 || dim == 2)


assert is more for testing, require ?

ok, changed

wzhongyuan · 2019-09-09T02:16:27Z

spark/dl/src/main/scala/com/intel/analytics/bigdl/transform/vision/image/util/BboxUtil.scala

+  private def decodeSignalBoxWithWeight(encodeBox: Tensor[Float], bbox: Tensor[Float],
+             weight: Array[Float], decodeBox: Tensor[Float]): Unit = {
+    require(bbox.nDimension() == 1 && encodeBox.nDimension() == 1 && decodeBox.dim() == 1,
+    s"Only support decode single bbox, but get ${bbox.nDimension()}, ${encodeBox.nDimension()}")


why missing decodebox dimension in output log ?

ok, changed

wzhongyuan · 2019-09-09T04:46:06Z

LGTM

zhangxiaoli73 · 2019-09-09T06:10:46Z

jenkins pass：
http://mlt-bddl01.sh.intel.com:8080/job/BigDL-PRVN-lite/844/

* add boxhead * add SerialTest * meet pr comments

* add conda to docker image * Update Dockerfile * Update Dockerfile Co-authored-by: glorysdj <glorysdj@gmail.com>

* add hyperzoo for k8s support (intel-analytics#2140) * add hyperzoo for k8s support * format * format * format * format * run examples on k8s readme (intel-analytics#2163) * k8s readme * fix jdk download issue (intel-analytics#2219) * add doc for submit jupyter notebook and cluster serving to k8s (intel-analytics#2221) * add hyperzoo doc * add hyperzoo doc * add hyperzoo doc * add hyperzoo doc * fix jdk download issue (intel-analytics#2223) * bump to 0.9s (intel-analytics#2227) * update jdk download url (intel-analytics#2259) * update some previous docs (intel-analytics#2284) * K8docsupdate (intel-analytics#2306) * Update README.md * Update s3 related links in readme and documents (intel-analytics#2489) * Update s3 related links in readme and documents * Update s3 related links in readme and documents * Update s3 related links in readme and documents * Update s3 related links in readme and documents * Update s3 related links in readme and documents * Update s3 related links in readme and documents * update * update * modify line length limit * update * Update mxnet-mkl version in hyper-zoo dockerfile (intel-analytics#2720) Co-authored-by: gaoping <pingx.gao@intel.com> * update bigdl version (intel-analytics#2743) * update bigdl version * hyperzoo dockerfile add cluster-serving (intel-analytics#2731) * hyperzoo dockerfile add cluster-serving * update * update * update * update jdk url * update jdk url * update Co-authored-by: gaoping <pingx.gao@intel.com> * Support init_spark_on_k8s (intel-analytics#2813) * initial * fix * code refactor * bug fix * update docker * style * add conda to docker image (intel-analytics#2894) * add conda to docker image * Update Dockerfile * Update Dockerfile Co-authored-by: glorysdj <glorysdj@gmail.com> * Fix code blocks indents in .md files (intel-analytics#2978) * Fix code blocks indents in .md files Previously a lot of the code blocks in markdown files were horribly indented with bad white spaces in the beginning of lines. Users can't just select, copy, paste, and run (in the case of python). I have fixed all these, so there is no longer any code block with bad white space at the beginning of the lines. It would be nice if you could try to make sure in future commits that all code blocks are properly indented inside and have the right amount of white space in the beginning! * Fix small style issue * Fix indents * Fix indent and add \ for multiline commands Change indent from 3 spaces to 4, and add "\" for multiline bash commands Co-authored-by: Yifan Zhu <fanzhuyifan@gmail.com> * enable bigdl 0.12 (intel-analytics#3101) * switch to bigdl 0.12 * Hyperzoo example ref (intel-analytics#3143) * specify pip version to fix oserror 0 of proxy (intel-analytics#3165) * Bigdl0.12.1 (intel-analytics#3155) * bigdl 0.12.1 * bump 0.10.0-Snapshot (intel-analytics#3237) * update runtime image name (intel-analytics#3250) * update jdk download url (intel-analytics#3316) * update jdk8 url (intel-analytics#3411) Co-authored-by: ardaci <dongjie.shi@intel.com> * update hyperzoo docker image (intel-analytics#3429) * update hyperzoo image (intel-analytics#3457) * fix jdk in az docker (intel-analytics#3478) * fix jdk in az docker * fix jdk for hyperzoo * fix jdk in jenkins docker * fix jdk in cluster serving docker * fix jdk * fix readme * update python dep to fit cnvrg (intel-analytics#3486) * update ray version doc (intel-analytics#3568) * fix deploy hyperzoo issue (intel-analytics#3574) Co-authored-by: gaoping <pingx.gao@intel.com> * add spark fix and net-tools and status check (intel-analytics#3742) * intsall netstat and add check status * add spark fix for graphene * bigdl 0.12.2 (intel-analytics#3780) * bump to 0.11-S and fix version issues except ipynb * add multi-stage build Dockerfile (intel-analytics#3916) * add multi-stage build Dockerfile * multi-stage build dockerfile * multi-stage build dockerfile * Rename Dockerfile.multi to Dockerfile * delete Dockerfile.multi * remove comments, add TINI_VERSION to common arg, remove Dockerfile.multi * multi-stage add tf_slim Co-authored-by: shaojie <shaojiex.bai@intel.com> * update hyperzoo doc and k8s doc (intel-analytics#3959) * update userguide of k8s * update k8s guide * update hyperzoo doc * Update k8s.md add note * Update k8s.md add note * Update k8s.md update notes * fix 4087 issue (intel-analytics#4097) Co-authored-by: shaojie <shaojiex.bai@intel.com> * fixed 4086 and 4083 issues (intel-analytics#4098) Co-authored-by: shaojie <shaojiex.bai@intel.com> * Reduce image size (intel-analytics#4132) * Reduce Dockerfile size 1. del redis stage 2. del flink stage 3. del conda & exclude some python packages 4. add copies layer stage * update numpy version to 1.18.1 Co-authored-by: zzti-bsj <shaojiex.bai@intel.com> * update hyperzoo image (intel-analytics#4250) Co-authored-by: Adria777 <Adria777@github.com> * bigdl 0.13 (intel-analytics#4210) * bigdl 0.13 * update * print exception * pyspark2.4.6 * update release PyPI script * update * flip snapshot-0.12.0 and spark2.4.6 (intel-analytics#4254) * s-0.12.0 master * Update __init__.py * Update python.md * fix docker issues due to version update (intel-analytics#4280) * fix docker issues * fix docker issues * update Dockerfile to support spark 3.1.2 && 2.4.6 (intel-analytics#4436) Co-authored-by: shaojie <otnw_bsj@163.com> * update hyperzoo, add lib for tf2 (intel-analytics#4614) * delete tf 1.15.0 (intel-analytics#4719) Co-authored-by: Le-Zheng <30695225+Le-Zheng@users.noreply.github.com> Co-authored-by: pinggao18 <44043817+pinggao18@users.noreply.github.com> Co-authored-by: pinggao187 <44044110+pinggao187@users.noreply.github.com> Co-authored-by: gaoping <pingx.gao@intel.com> Co-authored-by: Kai Huang <huangkaivision@gmail.com> Co-authored-by: GavinGu07 <55721214+GavinGu07@users.noreply.github.com> Co-authored-by: Yifan Zhu <zhuyifan@stanford.edu> Co-authored-by: Yifan Zhu <fanzhuyifan@gmail.com> Co-authored-by: Song Jiaming <litchy233@gmail.com> Co-authored-by: ardaci <dongjie.shi@intel.com> Co-authored-by: Yang Wang <yang3.wang@intel.com> Co-authored-by: zzti-bsj <2779090360@qq.com> Co-authored-by: shaojie <shaojiex.bai@intel.com> Co-authored-by: Lingqi Su <33695124+Adria777@users.noreply.github.com> Co-authored-by: Adria777 <Adria777@github.com> Co-authored-by: shaojie <otnw_bsj@163.com>

zhangxiaoli73 force-pushed the boxhead branch from 624d621 to 8becc60 Compare August 29, 2019 01:35

zhangxiaoli73 requested review from wzhongyuan and i8run August 29, 2019 02:30

zhangxiaoli73 force-pushed the boxhead branch from 604c54e to 3ae5019 Compare September 2, 2019 07:23

wzhongyuan reviewed Sep 5, 2019

View reviewed changes

i8run reviewed Sep 6, 2019

View reviewed changes

zhangxiaoli73 added 4 commits September 9, 2019 09:41

add boxhead

4f64aac

refactor

4dcf1e9

fix style check

9ffcf92

add SerialTest

fa42b1e

zhangxiaoli73 force-pushed the boxhead branch from c0b135d to df3038f Compare September 9, 2019 01:48

wzhongyuan reviewed Sep 9, 2019

View reviewed changes

meet pr comments

4ed6779

zhangxiaoli73 force-pushed the boxhead branch 2 times, most recently from 1754430 to 4ed6779 Compare September 9, 2019 03:05

zhangxiaoli73 merged commit ae33776 into intel-analytics:master Sep 9, 2019

zhangxiaoli73 added a commit to zhangxiaoli73/BigDL that referenced this pull request Oct 8, 2019

[New feature] Add Boxhead (intel-analytics#2894)

92206f1

* add boxhead * add SerialTest * meet pr comments

Le-Zheng pushed a commit to Le-Zheng/BigDL that referenced this pull request Oct 20, 2021

add conda to docker image (intel-analytics#2894)

68d2442

* add conda to docker image * Update Dockerfile * Update Dockerfile Co-authored-by: glorysdj <glorysdj@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[New feature] Add Boxhead #2894

[New feature] Add Boxhead #2894

zhangxiaoli73 commented Aug 22, 2019

wzhongyuan Sep 5, 2019

zhangxiaoli73 Sep 9, 2019

wzhongyuan Sep 5, 2019

zhangxiaoli73 Sep 9, 2019

wzhongyuan Sep 5, 2019

zhangxiaoli73 Sep 9, 2019

wzhongyuan Sep 5, 2019

zhangxiaoli73 Sep 9, 2019

wzhongyuan Sep 5, 2019

zhangxiaoli73 Sep 9, 2019

wzhongyuan Sep 5, 2019

zhangxiaoli73 Sep 9, 2019

wzhongyuan Sep 5, 2019

zhangxiaoli73 Sep 9, 2019

wzhongyuan Sep 5, 2019

zhangxiaoli73 Sep 9, 2019

wzhongyuan commented Sep 5, 2019

i8run Sep 6, 2019

zhangxiaoli73 Sep 9, 2019

wzhongyuan Sep 9, 2019

zhangxiaoli73 Sep 9, 2019

wzhongyuan Sep 9, 2019

zhangxiaoli73 Sep 9, 2019

wzhongyuan Sep 9, 2019

zhangxiaoli73 Sep 9, 2019

wzhongyuan commented Sep 9, 2019

zhangxiaoli73 commented Sep 9, 2019

[New feature] Add Boxhead #2894

[New feature] Add Boxhead #2894

Conversation

zhangxiaoli73 commented Aug 22, 2019

What changes were proposed in this pull request?

How was this patch tested?

Related links or issues (optional)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wzhongyuan commented Sep 5, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wzhongyuan commented Sep 9, 2019

zhangxiaoli73 commented Sep 9, 2019