Skip to content

Conversation

@kjnasdfkjnsadfkjnds
Copy link
Collaborator

@kjnasdfkjnsadfkjnds kjnasdfkjnsadfkjnds commented Dec 3, 2025

Essential Elements of an Effective PR Description Checklist

  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS ABOVE HAVE BEEN CONSIDERED.

Purpose

Add run_seed field to chat completion responses to enable sequence validation in decentralized inference
systems. This implements the foundation for Stage 1 sequence check validation as described in the inference
validation proposal.

When a user provides a seed parameter, vLLM now:

  1. Generates a deterministic run_seed by hashing user_seed + inference_id
  2. Uses the first 16 hex characters of run_seed to derive the actual sampling seed
  3. Returns run_seed in the logprobs field of the response

This allows validators to verify that generated tokens were sampled from the claimed model's probability
distribution.

Test Plan

# Run validation utils tests
pytest tests/entrypoints/openai/test_validation_utils.py -v

# Test with actual inference
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "facebook/opt-125m",
    "messages": [{"role": "user", "content": "Count to 5"}],
    "seed": 42,
    "logprobs": true,
    "top_logprobs": 5,
    "max_tokens": 20,
    "chat_template": "{{messages[0].content}}"
  }'

Test Result

All tests pass:

tests/entrypoints/openai/test_validation_utils.py::test_generate_run_seed PASSED
tests/entrypoints/openai/test_validation_utils.py::test_generate_run_seed_deterministic PASSED
tests/entrypoints/openai/test_validation_utils.py::test_generate_run_seed_different_inference_ids PASSED
tests/entrypoints/openai/test_validation_utils.py::test_generate_run_seed_none PASSED
tests/entrypoints/openai/test_validation_utils.py::test_compute_derived_seed PASSED
tests/entrypoints/openai/test_validation_utils.py::test_compute_derived_seed_deterministic PASSED
tests/entrypoints/openai/test_validation_utils.py::test_compute_derived_seed_none PASSED

Response now includes run_seed:

{
  "choices": [{
    "logprobs": {
      "content": [...],
      "run_seed": "4ef75adbd0ec3fba28ade0cf79ba5d4afea742e476567517d348836bb1876c9b"
    }
  }]
}

(Optional) Documentation Update

None required, this is an additive change that adds an optional field to existing responses

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants