Add support for negative prompts

[allo-]

Is your feature request related to a problem? Please describe.
Some forks stable diffusion added support for negative prompts.

I tested this in the StableDiffusionPipeline and it seems to work that way with diffusers as well. I currently don't see a way to implement it without creating an own Pipeline, so it would be useful when it could be added to the official pipelines, so downstream projects can use it without an own pipeline class.

Describe the solution you'd like
The implementation from the link is straightforward. Add a negative_prompt variable to __call__ and change uncond_input

-                [""] * batch_size, padding="max_length", max_length=max_length, return_tensors="pt"
+                [negative_prompt] * batch_size, padding="max_length", max_length=max_length, return_tensors="pt"

I'd create a pull request, but I think a clean implementation should probably at least do the length check from text_embeddings as well and there may be other things I've overlooked, as I mostly added what is documented in the link.

Describe alternatives you've considered

• For testing I created a copy of the Pipeline to use it in my code. It would be outdated when diffusers changes things. Subclassing it wouldn't be easy as the change is in the middle of a method.
• Implementing it in the other two pipelines would probably be straightforward, but maybe this feature should wait for the UnifiedPipeline to avoid duplicate code.

[patrickvonplaten]

Hey @allo-,

Thanks a lot for the feature request - it looks indeed like a very cool feature! I don't think we'll have a UnifiedPipeline anytime soon. Could you open a PR maybe just on StableDiffusionPipeline that showcases how you could implement the feature?

This would be extremely useful :-)

[allo-]

I'll do later. I think duplicating the code into the other (torch) pipelines isn't hard either.

You are okay with me just adding another optional argument to __call__?

[patrickvonplaten]

Actually, sorry I just noticed there is already another PR open about this: #549 -> is there anything else we should support in your opinion? Or does this PR cover all of the use cases you were thinking about?

[allo-]

It looks fine to me. Two things I would have considered in my PR (but I didn't look at the other objects in detail, yet):

• Shouldn't the negative prompt have the same truncation warning as the positive one?
• Change the truncation warnings to
  • "The following part of your prompt was truncated because CLIP can only handle sequences up to"
  • "The following part of your negative_prompt was truncated because CLIP can only handle sequences up to"

In general, the negative prompt still uses ["string"]*batch_size and I still wanted to lookup if it shouldn't generate text_input_ids in a similiar way as the positive ones.
I may be wrong on this, as said I have only the very simple implementation yet and would have looked into the details later.

[allo-]

I guess there is nothing more to do here.