feat: ORT GenAI Stateful Compilation changes

[ankitm3k]

Description

This PR enables the essential features to enable ORT GenAI with OVEP using Stateful Compilation of ov::Model, inspired from OV GenAI pipeline flow.

I have introduced a new provider option enable_causallm which can be set to True for enabling the ORT GenAI pipeline with Causal LLM Models that are fully supported on OVEP in the custom config file called genai_config.json

Sample genai_config.json -

"provider_options": [
    {
         // Key "OpenVINO" is case sensitive and must be used as below as its defined by MSFT
         "OpenVINO":
          {
              "device_type": "NPU",
              // Mandatory provider option to be set always with ORT GenAI
              "enable_causallm" : "True",
              // (Applicable for NPU only) Optional setting for compilation with custom MAX_PROMPT_LEN & MIN_RESPONSE_LEN
              "load_config": "{\"NPU\":{\"MAX_PROMPT_LEN\":\"2048\",\"MIN_RESPONSE_LEN\":\"512\"}}"                            
                       
           }
    }
    ]

FYI the GenAI models in ONNX format are usually Stateless in nature & require dynamic shapes compilation.

[preetha-intel]

Can this condition be simplified Since this is not valid only for a dynamic model on NPU

[ankitm3k]

fixed

[RyanMetcalfeInt8]

Note that we may eventually need to support caching position_ids / logits which have more complicated shapes than just [1, <num_input_tokens>].

[ankitm3k]

will handle that logic in the next PR

[preetha-intel]

LGTM

[Copilot]

Pull Request Overview

This PR introduces stateful compilation support for ORT GenAI using OpenVINO by integrating a new provider option, enable_causallm, along with several supporting changes in compilation, inference request handling, and backend communication. Key changes include adding stateful model transformation utilities, updating the OpenVINO interface to support causal LM functionality, and modifying test cases and backend management to incorporate KV cache rewind and dynamic shapes handling.

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.

File	Description
onnxruntime/test/perftest/ort_test_session.cc	Extended option parsing for enable_causallm with boolean value checking and error messaging
onnxruntime/core/providers/openvino/ov_interface.{h,cc}	Added StatefulCompileModel mechanism and updated OVExeNetwork to carry extra stateful attributes
onnxruntime/core/providers/openvino/openvino_provider_factory.cc	Integrated parsing logic for enable_causallm and adjusted dynamic shapes flags for NPU devices
onnxruntime/* (multiple backend and execution provider files)	Updated backend and infer request handling to support KV cache operations, stateful inference, and additional configuration for ORT GenAI

Comments suppressed due to low confidence (2)

onnxruntime/core/providers/openvino/backends/basic_backend.h:54

• It would be helpful to add a comment explaining why tensor names such as ''beam_idx'', ''past_key_values'', and ''present'' are being skipped when session_context.enable_causallm is true. This aids future maintainers in understanding the rationale behind bypassing KV cache tensor mapping in stateful model scenarios.

if ((onnx_name.empty() || onnx_name == "beam_idx" ||

onnxruntime/core/providers/openvino/ov_interface.h:90

• [nitpick] Consider renaming the member variable 'compiled_model_obj' to simply 'compiled_model' for improved clarity and consistency with standard naming conventions.

ov::CompiledModel compiled_model_obj;