Compatibility issue with vllm 0.32

Describe the issue as clearly as possible:

A recent merge into vllm broke the integration with outlines. The deepcopy introduced does not work with outlines due to the dictionaries nested. This throws the following error:

TypeError: cannot pickle 'dict_values' object

vllm-project/vllm#2881

Steps/code to reproduce the bug:

Running vllm with the latest pull of vllm 

python -m outlines.serve.serve --model="mistralai/Mistral-7B-Instruct-v0.2"

Expected result:

vllm running normally

Error message:

INFO:     127.0.0.1:40000 - "POST /generate HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 29, in _raise_exception_on_finish
    task.result()
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 411, in run_engine_loop
    has_requests_in_progress = await self.engine_step()
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 382, in engine_step
    await self.engine.add_request_async(**new_request)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 241, in add_request_async
    return self.add_request(
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 468, in add_request
    sampling_params = copy.deepcopy(sampling_params)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 206, in _deepcopy_list
    append(deepcopy(a, memo))
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/azureuser/miniconda3/envs/vllm/lib/python3.10/copy.py", line 161, in deepcopy
    rv = reductor(4)
TypeError: cannot pickle 'dict_values' object

Outlines/Python version information:

Version information

``` (command output here) ```

Context for the issue:

No response

[MotzWanted]

I'm also having this issue. Are you having any concerns downgrading vllm to v0.3.1, and, thus, not serving a model with defensive sampling?

[lapp0]

vLLM is working to integrate outlines vllm-project/vllm#2819. Once they have it integrated hopefully this kind of error won't pop up and the systems will retain compatability.

To resolve this we can simply apply list() to RegexFSM.vocabulary: https://github.com/outlines-dev/outlines/blob/main/outlines/fsm/fsm.py#L124

>>> rfsm = RegexFSM("a", tokenizer)
>>> copy.deepcopy(rfsm)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nix/store/y027d3bvlaizbri04c1bzh28hqd6lj01-python3-3.11.7/lib/python3.11/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/y027d3bvlaizbri04c1bzh28hqd6lj01-python3-3.11.7/lib/python3.11/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/y027d3bvlaizbri04c1bzh28hqd6lj01-python3-3.11.7/lib/python3.11/copy.py", line 146, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/nix/store/y027d3bvlaizbri04c1bzh28hqd6lj01-python3-3.11.7/lib/python3.11/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/y027d3bvlaizbri04c1bzh28hqd6lj01-python3-3.11.7/lib/python3.11/copy.py", line 161, in deepcopy
    rv = reductor(4)
         ^^^^^^^^^^^
TypeError: cannot pickle 'dict_values' object
>>> rfsm.vocabulary = list(rfsm.vocabulary)
>>> copy.deepcopy(rfsm)
<outlines.fsm.fsm.RegexFSM object at 0x7f601eb78a10>

Good first PR for anyone looking to start contributing!

[lapp0]

For CFGFSM, we might choose a hacky solution for now that resolves this issue, otherwise trying to deepcopy a CFGFSM results in

TypeError: cannot pickle 'module' object

The hacky solution would be to construct a brand new CFGFSM from the start state (not a true deepcopy).

    def deepcopy(self) -> "CFGFSM":
        return copy(self)

[saattrupdan]

I've encountered this issue as well and have attempted a fix here: #711

@p-usefulbird Can you test and see if that fixes it?