Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pre-import api, config, and lovelace in bootstrap to avoid loading them at runtime #111752

Merged
merged 14 commits into from
Feb 29, 2024
Merged

Pre-import api, config, and lovelace in bootstrap to avoid loading them at runtime #111752

merged 14 commits into from
Feb 29, 2024

Conversation

bdraco
Copy link
Member

@bdraco bdraco commented Feb 28, 2024 •
edited
Loading

Proposed change

Pre-import api, config, and lovelace in bootstrap to avoid loading them at runtime when importing them will block the running loop.

We do this ahead of time to avoid a thundering herd of executor jobs to import frontend deps

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

@bdraco
Import frontend and deps in the executor
bd726e5 
Since we do not know which one is going to get loaded first
and is likely to import a large chunk of them we mark them
all for import in the executor. Once one of them loads the bulk
of them the loader is smart enough to know not to make new executor
jobs because it checks `sys.modules`
@bdraco
Copy link
Member Author

bdraco commented Feb 28, 2024 •
edited
Loading

The problem with this approach is sometimes we lose the race and import too many in the executor and overload it

Need to improve the loader checks first

bdraco
bdraco commented Feb 28, 2024
View reviewed changes
homeassistant/bootstrap.py Outdated Show resolved Hide resolved
bdraco
bdraco commented Feb 28, 2024
View reviewed changes
homeassistant/bootstrap.py Outdated Show resolved Hide resolved
@bdraco bdraco changed the title Import frontend and deps in the executor Pre-import config and lovelace in bootstrap to avoid loading them at runtime Feb 28, 2024
@bdraco bdraco added this to the 2024.3.0 milestone Feb 28, 2024
@bdraco bdraco marked this pull request as ready for review February 28, 2024 21:02
@bdraco bdraco requested a review from a team as a code owner February 28, 2024 21:02
@bdraco
Copy link
Member Author

bdraco commented Feb 28, 2024

It would have been good to do api as well but it's a circular import. Will need to fix that first

bdraco added a commit that referenced this pull request Feb 28, 2024
@bdraco
Move DATA_LOGGING constant to homeassistant.const
6ce4d2d 
In #111752 we want to pre-import api so its loaded before the
event loop is started to avoid blocking it. Currently its a
circular import that is blocking this change.
@bdraco bdraco mentioned this pull request Feb 28, 2024
Merged
20 tasks
@bdraco bdraco marked this pull request as draft February 28, 2024 23:52
@bdraco bdraco changed the title Pre-import config and lovelace in bootstrap to avoid loading them at runtime Pre-import api, config, and lovelace in bootstrap to avoid loading them at runtime Feb 29, 2024
@bdraco bdraco marked this pull request as ready for review February 29, 2024 00:11
@bdraco
Copy link
Member Author

bdraco commented Feb 29, 2024

trigger test failure is already fixed on dev

@bdraco bdraco merged commit 9f8fbb7 into dev Feb 29, 2024
47 of 49 checks passed
@bdraco bdraco deleted the frontend_import branch February 29, 2024 01:40
balloob pushed a commit that referenced this pull request Feb 29, 2024
@bdraco @balloob
Pre-import api, config, and lovelace in bootstrap to avoid loading th…
acfd1c2 
…em at runtime (#111752)
@github-actions github-actions bot locked and limited conversation to collaborators Mar 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@balloob balloob balloob approved these changes

Assignees
No one assigned
Labels
cherry-picked cla-signed code-quality core small-pr PRs with less than 30 lines.
Projects
None yet
Milestone
2024.3.0
Development

Successfully merging this pull request may close these issues.
2 participants
@bdraco @balloob