Arcface loss是不是比softmax loss更难收敛呢？

[Edwardmark]

No description provided.

[Lucien7786]

same confused and two questions to be discussed.
using #214 https://github.com/deepinsight/insightface/issues/214 2-stage pipeline to train mobilefacenet(y1)

1. using softmax loss can easily reached [lfw]99.0+, [cfp_fp]90+-, [agedb_30]90+. [train-acc]90+
   when using arcface loss, log shows:

---

INFO:root:Epoch[13] Batch [1360] Speed: 1534.20 samples/sec acc=0.122125
INFO:root:Epoch[13] Batch [1380] Speed: 1586.47 samples/sec acc=0.120125
lr-batch-epoch: 0.1 1385 13
testing verification..
(12000, 256)
infer time 6.56924
calculate_roc: tpr 0.985 fpr 0.00933333333333 accuracy 0.987833333333 best_thresholds 1.42
positive num : 3000.0 negative num : 3000.0
recall : 0.985
percision : 0.990613476366
[lfw][96000]XNorm: 16.530636 time: 1533611198
[lfw][96000]Accuracy-Flip: 0.98783+-0.00563
testing verification..
(14000, 256)
infer time 8.039899
calculate_roc: tpr 0.828571428571 fpr 0.100571428571 accuracy 0.864 best_thresholds 1.654
positive num : 3500.0 negative num : 3500.0
recall : 0.828571428571
percision : 0.891758917589
[cfp_fp][96000]XNorm: 13.877584 time: 1533611211
[cfp_fp][96000]Accuracy-Flip: 0.86400+-0.02022
testing verification..
(12000, 256)
infer time 6.542497
calculate_roc: tpr 0.885666666667 fpr 0.0563333333333 accuracy 0.914666666667 best_thresholds 1.519
positive num : 3000.0 negative num : 3000.0
recall : 0.885666666667
percision : 0.940198159943
[agedb_30][96000]XNorm: 16.119858 time: 1533611221
[agedb_30][96000]Accuracy-Flip: 0.91467+-0.01668
saving 48
INFO:root:Saved checkpoint to "../model-relu-y1-4/model-0048.params"
[96000]Accuracy-Highest: 0.92133

---

2. the second stage is based on the pretrained softmax model, load pretrained model success, but the train-acc seems starts from almost zero. Am I abused the two stage pipline method? waiting for some tips.

here is my two stage pipline on mobilefacenet(y1), almost followed by #214
NETWORK=y1
MODELDIR="../model-relu-$NETWORK"
mkdir -p "$MODELDIR"
PREFIX="$MODELDIR/model"
LOGFILE="$MODELDIR/log"

-----------first stage using softmax loss

LRSTEPS='240000,360000,440000'
CUDA_VISIBLE_DEVICES='0,1,2,3' python -u train_softmax.py --data-dir $DATA_DIR --network "$NETWORK" --loss-type 0 --prefix "$PREFIX" --per-batch-size 200 --lr-steps "$LRSTEPS" --margin-s 32.0 --margin-m 0.1 --ckpt 2 --emb-size 256 --fc7-wd-mult 10.0 --wd 0.00004 --max-steps 140020 > "$LOGFILE" 2>&1 &

--------------sencond stage switch to arcface loss

LRSTEPS='100000,160000,200000'
CUDA_VISIBLE_DEVICES='0,1,2,3' python -u train_softmax.py --data-dir $DATA_DIR --pretrained "$MODELDIR/model,70" --network "$NETWORK" --loss-type 4 --prefix "$PREFIX" --per-batch-size 200 --lr 0.1 --lr-steps "$LRSTEPS" --margin-s 64.0 --margin-m 0.5 --ckpt 2 --emb-size 256 --fc7-wd-mult 10.0 --wd 0.00004 > "$LOGFILE" 2>&1 &

[mehrazi]

@nttstar I faced with the same issue

[Vietdung113]

could you provide your solution?

[rocinant]

why is the question closed without answering, I faced exactly same problem. The acc stacked at about 0.3 during second stage after lots of iterations.

[leidaguo]

I have same question

[LucasWangZH]

Based on my implementation and experiments, ArcFace is definitely much more difficult to converge than using softmax loss. In my case, it's more difficult than CosFace. I dont know if someone has the same situation as mine.