Skip to content

Commit 66863b4

Browse files
skipped test for nightly failure (vllm-project#120)
SUMMARY: Skips test causing nightly failure for the short term - https://github.com/neuralmagic/nm-vllm/actions/runs/8257372912/job/22587806623#step:9:1296 Issue is related to an experimental feature called guided generation. Upstream team is working on resolving via vllm-project#3383. Change will cause merge conflicts with upstream. During next upstream sync where they have fixed the issue, we can turn this test back on TEST PLAN: Automation
1 parent 089773e commit 66863b4

File tree

1 file changed

+7
-0
lines changed

1 file changed

+7
-0
lines changed

tests/entrypoints/test_openai_server.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -534,6 +534,7 @@ async def test_guided_json_chat(server, client: openai.AsyncOpenAI):
534534
assert json1["name"] != json2["name"]
535535
assert json1["age"] != json2["age"]
536536

537+
537538
async def test_guided_regex_completion(server, client: openai.AsyncOpenAI):
538539
completion = await client.completions.create(
539540
model=MODEL_NAME,
@@ -550,6 +551,12 @@ async def test_guided_regex_completion(server, client: openai.AsyncOpenAI):
550551
assert re.fullmatch(TEST_REGEX, completion.choices[i].text) is not None
551552

552553

554+
# *** UPSTREAM SYNC ***
555+
# This test covers an experimental feature in vLLM, guided generation.
556+
# Currently, there is an upstream issue being debugged
557+
# See: https://github.com/vllm-project/vllm/pull/3383
558+
# Once this is resolved upstream, turn the test back on.
559+
@pytest.mark.skip("Issue upstream, currently being resolved.")
553560
async def test_guided_regex_chat(server, client: openai.AsyncOpenAI):
554561
messages = [{
555562
"role": "system",

0 commit comments

Comments
 (0)