Skip to content

Making produce/consume bi-directional #4775

Closed

Description

I'd like to suggest a change to produce and consume. I don't understand Julia's tasks in any depth, so this may not be practical, but I have used the code below and it does appear to work as expected.

The change is motivated by python's co-routines, which only have the equivalent of produce and consume (there's no yieldto), yet seem to be sufficiently general for many tasks.

The extension is pretty simple - it extends consume so that it can take an argument that is "returned" to the task being consumed, and it extends produce so that value is returned inside the task. This makes it possible to have a bi-directional "conversation" between producer and consumer. It remains asymmetric (one half calls produce, the other consume), so it's not as elegant as yieldto (which remains, of course) in some respects, but it allows the programmer to get a bit more general without needing new concepts.

Here is the implementation I have been using:

function produce2(v)
    ct = current_task()
    q = ct.consumers
    if isa(q,Condition)
        # make a task waiting for us runnable again
        notify1(q)
    end
    r = yieldto(ct.last, v)
    ct.parent = ct.last  # always exit to last consumer
    r
end
produce2(v...) = produce2(v)

function consume2(P::Task, args...)
    while !(P.runnable || P.done)
        if P.consumers === nothing
            P.consumers = Condition()
        end
        wait(P.consumers)
    end
    ct = current_task()
    prev = ct.last
    ct.runnable = false
    v = yieldto(P, args...)
    ct.last = prev
    ct.runnable = true
    if P.done
        q = P.consumers
        if !is(q, nothing)
            notify(q, P.result)
        end
    end
    v
end

which has only a handful of lines modified from the original (adding r in produce and args in consume).

Note that this does not break existing code (as far as I can tell) because currently no return is expected from produce, and consume doesn't accept additional parameters.

Finally two examples. First, some simple Julia code that allows a cipher to be implemented as a coroutine - it receives a character of plaintext and returns the encrypted value. https://github.com/andrewcooke/Stupid.jl/blob/master/src/Cipher.jl#L112

Second, a blog post on how this is useful in Python when simulating protocols between actors http://acooke.org/cute/UsingCorou0.html

Thanks,
Andrew

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    help wantedIndicates that a maintainer wants help on an issue or pull request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions