[Speculative decoding] Fix async executing

[zxdvd]

I'm try with speculative decoding and got problems with async engine. I think this fix it.
@cadedaniel Could you help to review this?

[cadedaniel]

Will take a look today

[cadedaniel]

Hi @zxdvd , thanks for the PR!

• Can you add the stacktrace of the crash you saw to the PR description?
• Can you add a test for async engine + spec decode, so we catch this in the future?

[cadedaniel]

I put a suggestion here for how to do the async test #4165 (comment)

[comaniac]

@zxdvd thanks for the timely PR! We also need this fix to unblock performance evaluation of #4353 so it'd be better to get it merged ASAP. Did you get a chance working on this PR?

[cadedaniel]

I think this one is closer #4165 (edited with right link)

[zxdvd]

I'll close this since it is duplicated to #4165