Make it easier to get the final output only

[evilstreak]

To get just the final output from a model that produces progressive output requires something like this:

import replicate
model = replicate.model(“kuprel/min-dalle”)
output = list(model.predict(“a black swan”))[-1]

That’s a bit of a handful, but the bigger issue is that it’s really not obvious.

You can’t call [-1] on the iterator because it’s not a list, so Python doesn’t know how to index from the end. Knowing you have to convert it to a list first seems non-obvious to Python newcomers, which some of our users will be.

Worse than that though, IMO, is that you have to know that the model generates progressive output. You need different code depending on whether or not the mode produces an iterator, and it’s not easy to find that information. Even if we made it easier I think it’d be clearer to explicitly state whether you want intermediate outputs or not, and to give you either an iterator or a single output accordingly.

If you want a single output from a model with an iterator, the library could just throw away the other results. If you want an iterator from a model that has none, the library could wrap it for you. That way the author is saying how they want to consume it, rather than adjusting their code to the model.

This’ll also help when models change between versions, either adding or removing progressive output, as previously working code won’t break with errors.

Data

• https://discord.com/channels/775512803439280149/969398139314384956/1000026922128003123
• There’s more, but I don’t have the links!

[evilstreak]

Another one: https://discord.com/channels/775512803439280149/969398139314384956/996429894445715566

[bfirsh]

I wonder if we could we solve this with better code examples on Replicate.

[evilstreak]

We could mitigate it with better examples, but I'm not sure it would solve it.

From earlier conversations I get the sense you're not keen on providing an interface like this in the API client. If that's right, can you elaborate on why? I think we might have a different philosophy of design here and I'd like to understand where and why they differ!

[bfirsh]

I have no particularly feelings on this, just generally exploring the design.

In general, it’s best to keep API designs simple and clean, so if we can improve the underlying architecture or documentation such that we don’t have to make the design more complex, that’s a win. In this example, the current design is not intuitive, but maybe there’s another language feature we can lean on such that this is possible in an intuitive way without increasing API surface area. But obviously if the optimal solution is adding to the API then we should do that.

That’s where this is coming from — just exploring the design.

[mattt]

This is resolved by #79.