Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lower memory overhead #11156

Open
pnico opened this issue May 9, 2023 · 1 comment
Open

Lower memory overhead #11156

pnico opened this issue May 9, 2023 · 1 comment
Labels
type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@pnico
Copy link

pnico commented May 9, 2023

Just importing many of these libraries has memory overhead that is large enough to cause problems in production especially in memory-constrained environments such as F1 instances.

Appengine customers are being directed to use these libraries for everything, including simply for getting app logs that are correlated per request (even though it's almost as easy to do this without using the python-logging library at all). Ideally this would be fine, but unfortunately the libraries are not designed in a way that allows selective or lazy imports, if you import one thing you bring in most of the library. If this increased memory overhead causes your instances to be killed, you might then need to increase the instance size and consequently your application's operating costs.

The discovery-based API client is being neglected on purpose, with long-standing bugs remaining unfixed, presumably to encourage everyone to use these libraries instead. We would like to use them, but we can't because they use too much memory.

googleapis/python-logging#623 (comment)

@parthea parthea changed the title Memory cost of import: pay for what you use Add support for lazy import May 9, 2023
@parthea parthea added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label May 9, 2023
@pnico
Copy link
Author

pnico commented Jun 22, 2023

I didn't intend this issue to be specifically about lazy imports, that was just one of the possible things I mentioned that might address the memory overhead issue. My intention here was just to point out the memory overhead.

If you are planning on implementing lazy loading in any case, feel free to link this issue with the one you make for that.

@pnico pnico changed the title Add support for lazy import Lower memory overhead Jun 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

2 participants