Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace grab with internal download function #5020

Merged
merged 8 commits into from
Sep 27, 2024
Merged

Replace grab with internal download function #5020

merged 8 commits into from
Sep 27, 2024
+1,213 −60

Commits on Sep 26, 2024

  1. Introduce internal/http.Download 

    This is a simple HTTP download function that can be used to download
stuff from the Internet. Special care is taken to detect and sanitize
server-suggested file names. Intended to replace the dependency on grab.

Signed-off-by: Tom Wieczorek <twieczorek@mirantis.com>
    @twz123
    twz123 committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    f25761b View commit details
    Browse the repository at this point in the history

  2. Remove unused logger from Autopilot Downloader 

    Signed-off-by: Tom Wieczorek <twieczorek@mirantis.com>
    @twz123
    twz123 committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    8b8d8c5 View commit details
    Browse the repository at this point in the history

  3. Remove httpResponse from downloader struct 

    It may very well be a local variable instead.

Signed-off-by: Tom Wieczorek <twieczorek@mirantis.com>
    @twz123
    twz123 committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    062cdbf View commit details
    Browse the repository at this point in the history

  4. Replace grab with internal download function 

    This makes the download process use a temporary file and atomic file
replacements on success. Allows existing files to be re-downloaded,
which was not possible before. On the contrary, it will re-download files
even if they didn't change.

Signed-off-by: Tom Wieczorek <twieczorek@mirantis.com>
    @twz123
    twz123 committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    b2b3bf9 View commit details
    Browse the repository at this point in the history

  5. Introduce k0scontext.WithInactivityTimeout 

    Returns a context that times out after a specified period of inactivity.
It provides a "keep alive" mechanism to reset the timeout based on
recent activity, ensuring that the context will remain valid for as long
as there is activity.

Signed-off-by: Tom Wieczorek <twieczorek@mirantis.com>
    @twz123
    twz123 committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    a657928 View commit details
    Browse the repository at this point in the history

  6. Introduce internal/io.WriterFunc 

    Moves the private implementation from the integration tests into the
regular code base. This adapter is useful from time to time.

Signed-off-by: Tom Wieczorek <twieczorek@mirantis.com>
    @twz123
    twz123 committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    ee3240a View commit details
    Browse the repository at this point in the history

  7. Implement cancellation of stale downloads 

    Cancel downloads whenever there's no data flowing for a certain amount
of time. Use k0scontext.WithInactivityTimeout for that.

Signed-off-by: Tom Wieczorek <twieczorek@mirantis.com>
    @twz123
    twz123 committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    1a54f6c View commit details
    Browse the repository at this point in the history

  8. Add a very long overall timeout for Autopilot downloads 

    This will ensure that the download will fail at some point, even if the
remote server is artificially slow.

Signed-off-by: Tom Wieczorek <twieczorek@mirantis.com>
    @twz123
    twz123 committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    8cb867f View commit details
    Browse the repository at this point in the history