Support multiple additions to ModelStream

[shawnz]

Currently when using the streaming feature, multiple successive additions to the ModelStream object will overwrite each other. This change adds the grammars together instead to preserve each addition.

For example, consider the following code:

import guidance
import guidance.models
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig


def main():
    model_id = "HuggingFaceH4/zephyr-7b-beta"
    quantization_config = BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_compute_dtype=torch.bfloat16,
        bnb_4bit_use_double_quant=True,
    )
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForCausalLM.from_pretrained(
        model_id, quantization_config=quantization_config
    )

    prompt = "Who was the first president of the United States? It was "

    print("Without streaming, but with multiple additions:")
    lm1 = guidance.models.Transformers(model=model, tokenizer=tokenizer)
    lm1 += prompt
    lm1 += guidance.gen(max_tokens=3)
    print(str(lm1))

    print("\nWith streaming and single addition:")
    lm2 = guidance.models.Transformers(model=model, tokenizer=tokenizer).stream()
    lm2 += prompt + guidance.gen(max_tokens=3)
    *_, last_lm2 = lm2
    print(str(last_lm2))

    print("\nWith streaming and multiple additions:")
    lm3 = guidance.models.Transformers(model=model, tokenizer=tokenizer).stream()
    lm3 += prompt
    lm3 += guidance.gen(max_tokens=3)
    *_, last_lm3 = lm3
    print(str(last_lm3))


if __name__ == "__main__":
    main()

Prior to this change, only the first two generations will give sensical output whereas the third will output garbage.

[slundberg]

Thanks @shawnz ! Can't believe that bug was still there...good catch :)