Skip to content

Can we account for stack size in our reported memory usage? #52523

Open

Description

If you want to know how much RSS / resident memory julia is consuming, you can get a reasonable estimate by asking for live bytes tracked by the GC + bytes allocated for codegen. However, one large missing gap is the memory that is mmapped for task stacks, only parts of which may be resident, based on how much of the stack its owning Task has used.

Can we please add accounting that measures the resident Task stack sizes once in a while and updates some internal accounting? Assuming this is expensive, we could do it only once in a while, maybe at the end of a full GC and no more frequently than 1x per minute or something?

We were thinking on linux we could probably do this by iterating all stacks and calling mincore to report how much memory is resident? And calling that from the end of a GC so that the julia threads are all still paused so it's safe to access the list of stacks?

(Is there an equivalent monitoring or observability label?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    observabilitymetrics, timing, understandability, reflection, logging, ...

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions