Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make local backup a backup agent #130623

Merged
merged 25 commits into from
Nov 18, 2024
Merged

Conversation

emontnemery
Copy link
Contributor

Proposed change

Make local backup a backup agent

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

To help with the load of incoming pull requests:

@home-assistant
Copy link

Hey there @home-assistant/core, mind taking a look at this pull request as it has been labeled with an integration (backup) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of backup can trigger bot actions by commenting:

  • @home-assistant close Closes the pull request.
  • @home-assistant rename Awesome new title Renames the pull request.
  • @home-assistant reopen Reopen the pull request.
  • @home-assistant unassign backup Removes the current integration label and assignees on the pull request, add the integration domain after the command.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

homeassistant/components/backup/backup.py Outdated Show resolved Hide resolved
homeassistant/components/backup/backup.py Show resolved Hide resolved
homeassistant/components/backup/backup.py Outdated Show resolved Hide resolved
class LocalBackupAgent(BackupAgent):
"""Define the format that backup agents can have."""

name = "local"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a separate PR, we may want to make name human readable and add a slug-type id.

if data_file := backup_file.extractfile("./backup.json"):
data = json_loads_object(data_file.read())
backup = LocalBackup(
id=cast(str, data["slug"]), # Do we need another ID?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The UploadedBackup base class says we need an ID, I'm not sure why, manager and frontend should care about the slug only.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The id is the agent's way of identifying the backup. The slug is something that the manager uses to identify the backup. They could be the same but I think it's good to separate terms since the semantics are different.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, but if we don't need the additional identifier there's no point in storing a reference to the slug twice.
The manager will only ever ask the agent to handle a backup based on the slug. The ID is an implementation detail which individual agents can add if they need one.

Copy link
Member

@MartinHjelmare MartinHjelmare Nov 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now the id is part of the interface and the id parameter is used to eg download a backup. We could remove it and force agents to keep track of backups via the slug.

I don't like the name slug since it doesn't describe the purpose of the parameter so I'd prefer to limit its use in interfaces.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.
I suggest we rename it to backup_id which makes the purpose clear.
Local agents can then add their own ID if needed for internal purpose.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect.

homeassistant/components/backup/manager.py Outdated Show resolved Hide resolved
homeassistant/components/backup/manager.py Outdated Show resolved Hide resolved
homeassistant/components/backup/manager.py Outdated Show resolved Hide resolved
homeassistant/components/backup/manager.py Outdated Show resolved Hide resolved
Comment on lines 292 to 293
async def async_get_backups(self, **kwargs: Any) -> dict[str, Backup]:
"""Return backups."""
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense the manager knows about all backups across all agents.
The manager should not have to ask each time though, there needs to be some caching. Also, either frontend or an agent should be able to initiate a refresh.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The caching needs to happen in the individual agents, they know what's reasonable, not the manager

homeassistant/components/backup/backup.py Outdated Show resolved Hide resolved
homeassistant/components/backup/backup.py Show resolved Hide resolved
homeassistant/components/backup/manager.py Outdated Show resolved Hide resolved
homeassistant/components/backup/manager.py Outdated Show resolved Hide resolved
homeassistant/components/backup/manager.py Outdated Show resolved Hide resolved
homeassistant/components/backup/backup.py Outdated Show resolved Hide resolved
homeassistant/components/backup/backup.py Outdated Show resolved Hide resolved
homeassistant/components/backup/manager.py Outdated Show resolved Hide resolved

return backup

async def async_remove_backup(self, *, slug: str, **kwargs: Any) -> None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't part of the agent interface yet. I'm working on adding async_delete_backup.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can keep it for now. I'll replace it later.

homeassistant/components/backup/backup.py Show resolved Hide resolved

if backup is None or not backup.path.exists():
if backup is None or path is None or not path.exists():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to run the exists check in the executor.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd replace this check with try... except FileNotFoundError when reading the file though since the file can be removed at any time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems aiohttp already checks that, we don't need to do it

@@ -42,12 +42,13 @@ async def get(

manager = cast(BackupManager, request.app[KEY_HASS].data[DATA_MANAGER])
backup = await manager.async_get_backup(slug=slug)
path = await manager.async_get_backup_path(slug=slug)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd expect us to download the backup from the agent here. If the agent already has it downloaded it doesn't need to do it again of course. The downloaded backup will always have a local path on disk.

homeassistant/components/backup/manager.py Outdated Show resolved Hide resolved
homeassistant/components/backup/manager.py Outdated Show resolved Hide resolved
homeassistant/components/backup/manager.py Outdated Show resolved Hide resolved
@epenet epenet marked this pull request as draft November 15, 2024 08:30
@emontnemery emontnemery marked this pull request as ready for review November 18, 2024 14:41
@@ -48,8 +49,10 @@ async def async_setup(hass: HomeAssistant, config: ConfigType) -> bool:

async def async_handle_create_service(call: ServiceCall) -> None:
"""Service handler for creating backups."""
agent_id = list(backup_manager.local_backup_agents)[0]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side note: We should probably handle missing local agents here and raise HomeAssistantError.

homeassistant/components/backup/backup.py Outdated Show resolved Hide resolved
homeassistant/components/backup/manager.py Outdated Show resolved Hide resolved
manager = hass.data[DATA_MANAGER]
await manager.load_platforms()

manager = cast(BackupManager, hass.data[DATA_MANAGER])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this change later or can we allow the base manager instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should review the BackupManager base class as part of adding a hassio backup manager, then this cast should be removed.

return result

local_agent = hass.data[DATA_MANAGER].backup_agents[LOCAL_AGENT_ID]
local_agent._backups = {backups.slug: backups for backups in backups}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
local_agent._backups = {backups.slug: backups for backups in backups}
local_agent._backups = {backup.slug: backup for backup in backups}

@emontnemery
Copy link
Contributor Author

emontnemery commented Nov 18, 2024

The test failure is unrelated to this PR

@emontnemery emontnemery merged commit a47a70d into allthebackupchanges Nov 18, 2024
59 of 60 checks passed
@emontnemery emontnemery deleted the local_backup_agent branch November 18, 2024 16:10
@emontnemery emontnemery mentioned this pull request Nov 18, 2024
19 tasks
@github-actions github-actions bot locked and limited conversation to collaborators Nov 19, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants