Skip to content

[multiple clients] Do not reference static or long-lived protobuf message objects from transient protobuf messages #28100

Open
@dazuma

Description

There is a known issue in protobuf (protocolbuffers/protobuf#10106) in which if transient message object A references a long-lived message object B, the protobuf data underlying A will not get garbage collected until B is collectable, even if A is not reachable. This is because A and B are forced into the same protobuf arena, which protobuf will not dispose as long as B is live. If many such transient objects A are created, this will present as a memory leak. (Note: this may be a simplification of the exact repro case and explanation; see the original protobuf issue for more details.)

The upshot is, client libraries should not hold onto any protobuf message objects that can be used as field values in other objects that are meant to be transient. A common example could be a "common" value held in a Ruby constant, which is then used as a field value in a client request object—this will cause all such request objects to remain in memory indefinitely, as long as the Ruby constant remains in scope.

Instead, any such common objects should either be created "on the fly" so they can be collected along with the object that references them, or "deep copied" so the copy actually referenced is collectable even if the original is long-lived.

We need to audit all handwritten client libraries and make this change. One known case is in the google-cloud-bigtable client: the Google::Cloud::Bigtable::RowFilter module includes several constants which are proto messages used in filter chains.

References:

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions