Skip to content

Implement lazy module imports for Python #2007

Closed

Description

Overview

We currently have ~12,000 files in our generated SDK. Python's default import mechanism recursively loads and caches all modules defined in the top level of the entry file, BaseGraphServiceClient in our case, which results in significant overhead of longer startup time and more memory usage. Lazy imports are one way to mitigate this, where we defer importing modules until they are explicitly used.

Potential benefits:

  1. Cut startup time by 90%
  2. Less memory usage
  3. Eliminate circular import errors
  4. Improved fault tolerance. Broken code will only affect related paths.

Work involved

  1. We need to implement a utility method called lazy_import that will make use of Python's LazyLoader to defer imports. This can be defined in abstractions library or in a module within the service library.
  2. Update code generation import structure to make use of the lazy import method. LazyLoader only supports absolute paths, so this would result in code like so:
# previous
from .admin import admin_request_builder

# modified
admin_request_builder = lazy_import('msgraph.generated.admin.admin_request_builder')

The admin_request_builder module would only ever be imported if a user calls the client.admin() path using the fluent interface.

We can extend this implementation beyond the api constructor class to the request builder and model classes to realize even more gains.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

Type

No type

Projects

  • Status

    Done ✔️

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions