List types set in web get stuck in "starting"

[venkr]

Union and list types were added as of #1311, however when you call them with an invalid type, the UI incorrectly displays them as "Booting" forever.

Test case:

• Use the following Replicate model: https://replicate.com/venkr/cog-union-test
• 1: Try setting string_union to a string -> it works fine, response is received in milliseconds.
• 2: Try setting string_union to a string, and string_list to a string -> the model gets stuck in booting forever.

I think it's fine if the UI doesn't support / breaks with this for now, but it should fail fast rather than showing the user that it's stuck in booting.

Some example prediction-ids:

• Success where an invalid type isn't provided: 4hfvd6bbr5ahfgakoe72ymcwxy
• Infinite booting where an invalid type is provided: jew4vajbowvtett2qm2djrwgiq

[venkr]

For reference, the source code of the test model:

from cog import BasePredictor, Input, Path
from typing import List, Union

class Predictor(BasePredictor):
    def setup(self):
        pass

    def predict(
        self,
        string_union: Union[str, List[str]] = Input(description="Either a string or a list of strings"),
        string_list: List[str] = Input(description="A list of strings or nothing", default=None)
    ) -> str:
        """Run a single prediction on the model"""
        if isinstance(string_union, str):
            print("Got a string")
        
        if isinstance(string_union, list):
            print("Got a list of strings")

        return "Success"

[technillogue]

this actually got fixed on the 12th by @mattrothenberg!