[Log][Bugfix] Fix default value check for image_url.detail

[mgoin]

The warning about not supporting image_url.detail is overzealous as it doesn't deal with detail == None, which can be the default case. This PR simply has None represented as "auto" by default, which matches OpenAI: https://github.com/openai/openai-python/blob/e1b2f8216cc69e802475a0d438e40e0e74510de4/src/openai/types/beta/threads/image_url.py#L18-L23

Example of annoying warning on requests:

WARNING 10-23 15:45:56 chat_utils.py:453] 'image_url.detail' is currently not supported and will be ignored.
INFO 10-23 15:45:56 logger.py:37] Received request chat-c1c8229b85814c1592ab53633e058c17: prompt: "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\n<|image|>Which contemporary artist's architectural style is resembled in the painting's setting? \n\nA. Amolfo di Cambio\nB. Brunelleschi\nC. Bernini\nD. Michelangelo\nAnalyze the image and question carefully, using step-by-step reasoning.\nFirst, describe any image provided in detail. Then, present your reasoning. And finally your final answer in this format:\nFinal Answer: <answer>\nwhere <answer> is:\n- The single correct letter choice A, B, C, D, E, F, etc. when options are provided. Only include the letter.\n- Your direct answer if no options are given, as a single phrase or number.\n- If your answer is a number, only include the number without any unit.\n- If your answer is a word or phrase, do not paraphrase or reformat the text you see in the image.\n- You cannot answer that the question is unanswerable. You must either pick an option or provide a direct answer.\nIMPORTANT: Remember, to end your answer with Final Answer: <answer>.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n", params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=2048, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), guided_decoding=None, prompt_token_ids: [128000, 128006, 882, 128007, 271, 128256, 23956, 19225, 10255, 596, 43563, 1742, 374, 96858, 304, 279, 19354, 596, 6376, 30, 4815, 32, 13, 3383, 337, 831, 1891, 34896, 822, 198, 33, 13, 35561, 37907, 14946, 198, 34, 13, 14502, 6729, 198, 35, 13, 45506, 89010, 198, 2127, 56956, 279, 2217, 323, 3488, 15884, 11, 1701, 3094, 14656, 30308, 33811, 627, 5451, 11, 7664, 904, 2217, 3984, 304, 7872, 13, 5112, 11, 3118, 701, 33811, 13, 1628, 5616, 701, 1620, 4320, 304, 420, 3645, 512, 19918, 22559, 25, 366, 9399, 397, 2940, 366, 9399, 29, 374, 512, 12, 578, 3254, 4495, 6661, 5873, 362, 11, 426, 11, 356, 11, 423, 11, 469, 11, 435, 11, 5099, 13, 994, 2671, 527, 3984, 13, 8442, 2997, 279, 6661, 627, 12, 4718, 2167, 4320, 422, 912, 2671, 527, 2728, 11, 439, 264, 3254, 17571, 477, 1396, 627, 12, 1442, 701, 4320, 374, 264, 1396, 11, 1193, 2997, 279, 1396, 2085, 904, 5089, 627, 12, 1442, 701, 4320, 374, 264, 3492, 477, 17571, 11, 656, 539, 63330, 10857, 477, 312, 2293, 279, 1495, 499, 1518, 304, 279, 2217, 627, 12, 1472, 4250, 4320, 430, 279, 3488, 374, 653, 9399, 481, 13, 1472, 2011, 3060, 3820, 459, 3072, 477, 3493, 264, 2167, 4320, 627, 99843, 25, 20474, 11, 311, 842, 701, 4320, 449, 13321, 22559, 25, 366, 9399, 14611, 128009, 128006, 78191, 128007, 271], lora_request: None, prompt_adapter_request: None.
INFO 10-23 15:45:56 async_llm_engine.py:207] Added request chat-c1c8229b85814c1592ab53633e058c17.
WARNING 10-23 15:45:56 chat_utils.py:453] 'image_url.detail' is currently not supported and will be ignored.
INFO 10-23 15:45:56 logger.py:37] Received request chat-7cf5ff396d3843b2b3ccf49a946e74b9: prompt: '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\n<|image|>What style characterizes the painting? \n\nA. Early Renaissance\nB. High Renaissance\nC. Mannerist\nD. Baroque\nAnalyze the image and question carefully, using step-by-step reasoning.\nFirst, describe any image provided in detail. Then, present your reasoning. And finally your final answer in this format:\nFinal Answer: <answer>\nwhere <answer> is:\n- The single correct letter choice A, B, C, D, E, F, etc. when options are provided. Only include the letter.\n- Your direct answer if no options are given, as a single phrase or number.\n- If your answer is a number, only include the number without any unit.\n- If your answer is a word or phrase, do not paraphrase or reformat the text you see in the image.\n- You cannot answer that the question is unanswerable. You must either pick an option or provide a direct answer.\nIMPORTANT: Remember, to end your answer with Final Answer: <answer>.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n', params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=2048, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), guided_decoding=None, prompt_token_ids: [128000, 128006, 882, 128007, 271, 128256, 3923, 1742, 3752, 4861, 279, 19354, 30, 4815, 32, 13, 23591, 55383, 198, 33, 13, 5234, 55383, 198, 34, 13, 386, 4992, 380, 198, 35, 13, 4821, 61652, 198, 2127, 56956, 279, 2217, 323, 3488, 15884, 11, 1701, 3094, 14656, 30308, 33811, 627, 5451, 11, 7664, 904, 2217, 3984, 304, 7872, 13, 5112, 11, 3118, 701, 33811, 13, 1628, 5616, 701, 1620, 4320, 304, 420, 3645, 512, 19918, 22559, 25, 366, 9399, 397, 2940, 366, 9399, 29, 374, 512, 12, 578, 3254, 4495, 6661, 5873, 362, 11, 426, 11, 356, 11, 423, 11, 469, 11, 435, 11, 5099, 13, 994, 2671, 527, 3984, 13, 8442, 2997, 279, 6661, 627, 12, 4718, 2167, 4320, 422, 912, 2671, 527, 2728, 11, 439, 264, 3254, 17571, 477, 1396, 627, 12, 1442, 701, 4320, 374, 264, 1396, 11, 1193, 2997, 279, 1396, 2085, 904, 5089, 627, 12, 1442, 701, 4320, 374, 264, 3492, 477, 17571, 11, 656, 539, 63330, 10857, 477, 312, 2293, 279, 1495, 499, 1518, 304, 279, 2217, 627, 12, 1472, 4250, 4320, 430, 279, 3488, 374, 653, 9399, 481, 13, 1472, 2011, 3060, 3820, 459, 3072, 477, 3493, 264, 2167, 4320, 627, 99843, 25, 20474, 11, 311, 842, 701, 4320, 449, 13321, 22559, 25, 366, 9399, 14611, 128009, 128006, 78191, 128007, 271], lora_request: None, prompt_adapter_request: None.
INFO 10-23 15:45:56 async_llm_engine.py:207] Added request chat-7cf5ff396d3843b2b3ccf49a946e74b9.
WARNING 10-23 15:45:56 chat_utils.py:453] 'image_url.detail' is currently not supported and will be ignored.
INFO 10-23 15:45:56 logger.py:37] Received request chat-a1052d9ee64d4f30807f18fd29f586da: prompt: '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\n<|image|>To which art historical period is the painting most closely associated? \n\nA. Baroque\nB. Realism\nC. Romanticism\nD. Impressionism\nAnalyze the image and question carefully, using step-by-step reasoning.\nFirst, describe any image provided in detail. Then, present your reasoning. And finally your final answer in this format:\nFinal Answer: <answer>\nwhere <answer> is:\n- The single correct letter choice A, B, C, D, E, F, etc. when options are provided. Only include the letter.\n- Your direct answer if no options are given, as a single phrase or number.\n- If your answer is a number, only include the number without any unit.\n- If your answer is a word or phrase, do not paraphrase or reformat the text you see in the image.\n- You cannot answer that the question is unanswerable. You must either pick an option or provide a direct answer.\nIMPORTANT: Remember, to end your answer with Final Answer: <answer>.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n', params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=2048, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), guided_decoding=None, prompt_token_ids: [128000, 128006, 882, 128007, 271, 128256, 1271, 902, 1989, 13970, 4261, 374, 279, 19354, 1455, 15499, 5938, 30, 4815, 32, 13, 4821, 61652, 198, 33, 13, 8976, 2191, 198, 34, 13, 76830, 2191, 198, 35, 13, 14727, 11433, 2191, 198, 2127, 56956, 279, 2217, 323, 3488, 15884, 11, 1701, 3094, 14656, 30308, 33811, 627, 5451, 11, 7664, 904, 2217, 3984, 304, 7872, 13, 5112, 11, 3118, 701, 33811, 13, 1628, 5616, 701, 1620, 4320, 304, 420, 3645, 512, 19918, 22559, 25, 366, 9399, 397, 2940, 366, 9399, 29, 374, 512, 12, 578, 3254, 4495, 6661, 5873, 362, 11, 426, 11, 356, 11, 423, 11, 469, 11, 435, 11, 5099, 13, 994, 2671, 527, 3984, 13, 8442, 2997, 279, 6661, 627, 12, 4718, 2167, 4320, 422, 912, 2671, 527, 2728, 11, 439, 264, 3254, 17571, 477, 1396, 627, 12, 1442, 701, 4320, 374, 264, 1396, 11, 1193, 2997, 279, 1396, 2085, 904, 5089, 627, 12, 1442, 701, 4320, 374, 264, 3492, 477, 17571, 11, 656, 539, 63330, 10857, 477, 312, 2293, 279, 1495, 499, 1518, 304, 279, 2217, 627, 12, 1472, 4250, 4320, 430, 279, 3488, 374, 653, 9399, 481, 13, 1472, 2011, 3060, 3820, 459, 3072, 477, 3493, 264, 2167, 4320, 627, 99843, 25, 20474, 11, 311, 842, 701, 4320, 449, 13321, 22559, 25, 366, 9399, 14611, 128009, 128006, 78191, 128007, 271], lora_request: None, prompt_adapter_request: None.
INFO 10-23 15:45:56 async_llm_engine.py:207] Added request chat-a1052d9ee64d4f30807f18fd29f586da.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[DarkLight1337]

Technically None is not allowed according to OpenAI spec, but I guess this makes it more robust.

[mgoin]

Oh I see, I was looking in the wrong place in the OpenAI library. The usage I saw this clear with was on mistral-evals, which seems to construct the requests manually so they just don't specify that entry https://github.com/mistralai/mistral-evals/blob/02bbf53fea669323010bba74f1b9874472d0d47c/eval/models.py#L64-L80