[INFERENCE] reduce lock contention in multi-stream shared inference

[walidbarakat]

Description

When multiple GStreamer pipelines share an InferenceImpl instance via model-instance-id, two lock-related issues cause stream starvation:

1. PushOutput() held output_frames_mutex while calling gst_pad_push() downstream via PushBufferToSrcPad(). Since gst_pad_push() can block (e.g. sync=true with clock wait, or downstream queue full), this held the mutex for 20-30ms per frame. Other streaming threads calling TransformFrameIp were blocked from pushing into output_frames, causing cascading starvation.

   Fix: Collect completed frames into a local vector under the lock, then release the lock before pushing them downstream. Also fixes head-of-line blocking by skipping incomplete frames on a per-stream basis rather than stopping iteration entirely.

2. TransformFrameIp held mutex for its entire duration (~13ms), serializing all streams through a single critical section. This mutex is unnecessary because every operation within the function is either per-element (not shared), per-buffer, or already protected by a finer-grained lock (output_frames_mutex, requests_mutex, freeRequests condvar).

   Fix: Remove _mutex from TransformFrameIp. The remaining use in UpdateObjectClasses is retained as it protects shared object_classes during multichannel reconfiguration.

How Has This Been Tested?

Tested with 21 concurrent streams sharing 7 model instances on Intel
Core Ultra 7 265K (2 P-cores, iGPU, batch-size=1): all streams
sustain 30fpshanges. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

batch_timeout (tested with 100ms) is needed as long as we use only 2 P-core to mitigate another shortcoming of low number of MC don't fill GPU queue fast enough.

general test can be performed using higher numbers of concurrent streams to get the highest can be processed using a certain number of model instances

Checklist:

• I agree to use the MIT license for my code changes.
• I have not introduced any 3rd party components incompatible with MIT.
• I have not included any company confidential information, trade secret, password or security token.
• I have performed a self-review of my code.