StagedTimer + TimerManager Impl + Pipeline.__call__ integration

[markurtz]

adds StagedTimer and TimerManager classes to enable easier benchmarking of pipelines and eventually the end to end deepsparse server flow. See docstring and Pipeline.__call__ changes for intended workflow.

example usage:
code:

from deepsparse import Pipeline
from deepsparse.loggers import PythonLogger

pipeline = Pipeline.create("text-classification", engine_type="onnxruntime", logger=PythonLogger())
pipeline("test")
print(pipeline.timer_manager)

output (after data logs):

21/06/2023 15:47:53:271368 Identifier: text_classification/prediction_latency/total_inference_seconds | Category: system | Logged Data: 0.05413583107292652 | Additional Info: {'pipeline_name': 'text_classification'}
21/06/2023 15:47:53:271383 Identifier: text_classification/prediction_latency/pre_process_seconds | Category: system | Logged Data: 0.0015975041314959526 | Additional Info: {'pipeline_name': 'text_classification'}
21/06/2023 15:47:53:271396 Identifier: text_classification/prediction_latency/engine_forward_seconds | Category: system | Logged Data: 0.0512517262250185 | Additional Info: {'pipeline_name': 'text_classification'}
21/06/2023 15:47:53:271408 Identifier: text_classification/prediction_latency/post_process_seconds | Category: system | Logged Data: 0.00011516967788338661 | Additional Info: {'pipeline_name': 'text_classification'}
TimerManager({'engine_forward': 0.0512517262250185, 'post_process': 0.00011516967788338661, 'pre_process': 0.0015975041314959526, 'total_inference': 0.05413583107292652})

[bfineran]

LGTM overall to initially land- two things we need to think through more:

• no clear design for nested pipelines as well as potentially timing bucketing
• eventually need to handle race conditions once we integrate into the server (not worried about server timings since we can call directly into pipeline.inference_timer from the fast api route)

[dbogunowicz]

@bfineran didn't we want to make a timer inherit from the Logger class? I am a bit confused with the current design.

[bfineran]

@bfineran didn't we want to make a timer inherit from the Logger class? I am a bit confused with the current design.

both work as long as we can split out the timer in scenarios where logger should also be decoupled such as the server. this meets that requirement.

[KSGulin]

LGTM