meaningless audio envelope in sp

[Liujingxiu23]

In synthesis process, breath is hard to control. So in the preprocess of the audios, I merge sp and ap as one phone named as sp, and set the audio sample value of sp as zero. When using other sing synthesis model, the synthesized audio at the position of sp is silence as I expected. But when using visinger2, meaningless audio envelope which seems as normal wave appears in location of sp.
See red boxes as the following images, the location of red boxes is sp.
Has anyone have the same problem and how to fix it?