[fix] inf2 pipeline fix for adapter unsupported #1927
Conversation
| found_adapters = found_adapter_per_item or found_adapters | ||
| if input_format_configs.is_adapters_supported: | ||
| adapters.extend(adapters_per_item) | ||
| found_adapters = found_adapter_per_item or found_adapters |
There was a problem hiding this comment.
Is found_adapters supposed to be doing something here on the right-hand side? It's set to False at the top of this function, and then this is its first use.
Just wanted to make sure this isn't some kind of bug that crept in at some point.
There was a problem hiding this comment.
@rohithkrn It looks like this was introduced in #1920. It looks like this was moved outside of the try catch per this discussion with David.
nit: I think adapters.extend and updating found_adapters should happen outside the try-except, to make it clearer that these state variables only get updated if we make it past the continue in except.
@davidthomas426 It looks like we are just getting a bool signaling if we have found an adapter or if it was set to true when calling this method, found_adapters is then passed to ParsedInput. I do think this is a little hard to read but it also isn't immediately clear what the rewrite would look like to me.
There was a problem hiding this comment.
yeah, I missed the check after moving outside the try block. found_adapters is used in the handler to determine whether adapter_data should be added to the Request object in the rolling batch inference. It is used to completely avoid processing adapters in the non-lora case. We can remove it as well set adapters for each request to None for non lora case.
Description
Add logic to ensure that when adapters are not supported that parse input still passes