-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
std/tasks: "small functions" optimization #443
Comments
This will probably only work with ARC/ORC but then so be it. |
I think std/tasks already requires either non-thread-local GC (so Boehm would work as well) or value types. Unless you use tasks in a non-multithreading context, but libraries that might want to do that (asyncdispatch/chronos) use closure iterators. |
In the current implementation, task is allocated in the stack and is cleaned up before We should probably pass proc invoke*(task: var Task) {.inline, gcsafe.} =
## Invokes the `task`.
assert task.callback != nil
if task.args == nil and task.destroy != nil:
task.callback(addr task.argsBuffer)
else:
task.callback(task.args) (though Task is always passed by reference internally) |
See discussion on Discord https://discord.com/channels/371759389889003530/768367394547957761/933307741462753300 The changes indeed work status-im/nim-taskpools@76a8593 but only if the spawner doesn't exit its scope before returning. AKA, structured parallelism:
One way to make it work is in proc `=sink`(dst: var Task, src: Task) =
`=destroy`(dst)
dst.callback = src.callback
dst.destroy = src.destroy
if src.args == cast[pointer](src.argsBuffer.addr):
# The arguments are stored inline the task
dst.argsBuffer = src.argsBuffer
dst.args = cast[pointer](dst.argsBuffer.addr)
else:
# The arguments are stored on the heap
dst.args = src.args This also implicitly assumes that Tasks are |
block:
var num = 0
proc hello(a: int) = inc num, a
let b = toTask hello(13)
b.invoke()
echo num
assert num == 13 doesn't work with POC
It generates c code like this: task is zero memoryed after toTask is called. Then task.argsBuffer become all zeros |
Currently std/tasks is implemented the following way:
Any function call with an argument will trigger an allocation. This is something that runtimes that use std/task can't avoid.
Given that runtimes also have to allocate a future/flowvar, scheduling a task always incur allocation overhead even if there are just 1 or 2 arguments that needs to be passed.[1]
GCC and Clang have an optimization where std:function can store up to 8 (?) bytes intrusive to the data structure and uses the heap otherwise.
It would significantly reduce memory overhead and memory fragmentation for long-running tasking runtimes if this was implemented:
Reclaiming memory allocated from another thread is non-trivial for allocators and requires synchronization and advanced techniques to avoid contention like extreme freelist sharding in the Mimalloc allocator
--
[1]: Runtime actually don't have to allocate a future/flowvar if those don't escape, they can use the caller stackframe if the compiler helps with heap allocation elision: (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2477r0.html#design-and-example, http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0981r0.html)
Proposal
For a to be decided size N, change the type to:
args
would point to either the buffer or the heap, change would be minimal and would mainly require changing this:to something in that vein.
at https://github.com/nim-lang/Nim/blob/d102b2f54c41f19a0545f28109d0a93550b5d886/lib/std/tasks.nim#L187
Discussion of the buffer size
The current task object has a size of 24 bytes.
proc(a, b): result
orproc(r, a, b)
, this would cover those efficiently.map
or a pointer to a result location)Working on large openarrays is often a very compelling reason to divide work in tasks.
Even if tasks were smaller, space would be unused.
The text was updated successfully, but these errors were encountered: