Skip to content

Interaction between addprocs and --heap-size-hint #50673

Closed

Description

I think this may be a feature request/issue report rather than a question, so I am posting here rather than on the discourse.

I am basically wondering: do Distributed.addprocs and the CLI flag --heap-size-hint interact with eachother, and, if so, how? If I specify --heap-size-hint to the head Julia process, and then dynamically create Julia processes with addprocs, how (if?) does it get split up among each of the processes created?

My current workaround is to pass --heap-size-hint to exeflags in each addprocs call, and divvy up the memory into smaller chunks, but this seems a bit hacky.

Just wondering about this because I've ran into various segfaults which I've never been able to track down (#47957). I realized that memory usage seems to explode when using multiple processes which are allocated dynamically, but the memory can be controlled by setting, e.g., --heap-size-hint when dynamically creating the workers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    docsThis change adds or pertains to documentationparallelismParallel or distributed computation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions