Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

agnos updater: set decompress max_length #33320

Merged

Conversation

andiradulescu
Copy link
Contributor

@andiradulescu andiradulescu commented Aug 17, 2024

This fixes the "MemoryError: Unable to allocate output buffer." when flashing big raw images. Write-up here.

The following changes have been implemented.

  • noop flashes until no more data is returned - this way any data that remains in self.buf after eof is detected gets flashed in every case - since the loop will call read until buffer is empty
  • decompress max_length is now set to length (max_length=-1 if not set) - so the output of decompress will never be more than the size of the return
  • setting b'' when needs_input=false - when it has more data to decompress after the previous length
  • if the last decompressed output has eof=True then we reached the end of file and the process stops

Flashing time for the sparse system image is similar (a little faster with max_length=length).

Flashing time for non-sparse raw 10GB image is ~5min.

Copy link
Contributor

github-actions bot commented Aug 17, 2024

Thanks for contributing to openpilot! In order for us to review your PR as quickly as possible, check the following:

  • Convert your PR to a draft unless it's ready to review
  • Read the contributing docs
  • Before marking as "ready for review", ensure:
    • the goal is clearly stated in the description
    • all the tests are passing
    • the change is something we merge
    • include a route or your device' dongle ID if relevant

system/hardware/tici/agnos.py Outdated Show resolved Hide resolved
system/hardware/tici/agnos.py Outdated Show resolved Hide resolved
system/hardware/tici/agnos.py Outdated Show resolved Hide resolved
@adeebshihadeh adeebshihadeh changed the title agnos: decompress max_length agnos updater: set decompress max_length Aug 17, 2024
@adeebshihadeh adeebshihadeh merged commit ba098f5 into commaai:master Aug 17, 2024
17 checks passed
@andiradulescu andiradulescu deleted the agnos_decompress_max_length branch August 18, 2024 06:55
@andiradulescu
Copy link
Contributor Author

good cleanup and thanks for applying!

Edison-CBS pushed a commit to Edison-CBS/openpilot that referenced this pull request Sep 15, 2024
* agnos: decompress max_length

* flash last chunk after eof

* don't decompress more than length

* cleanup

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: ba098f5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants