Skip to content

Add http & https asset sources (clean commit history) #17889

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 20 commits into
base: main
Choose a base branch
from

Conversation

mrchantey
Copy link
Contributor

@mrchantey mrchantey commented Feb 17, 2025

Objective

This is a duplicate of #16366 because I made a mess of the commit history. See that pr for more context and discussion.

Solution

  • Add http & https asset sources
  • Allows CDLA-Permissive-2.0 in deny.toml

Testing

  • Verify visually by running cargo run --example http_source --features="http_source"

Showcase

Bevy now supports assets loaded via url!

Add the http_source feature to load assets served over the web, with support for both native and wasm targets. On native targets the http_source_cache feature will use a basic caching mechanism to avoid repeated network requests.

  // Simply use a url where you would normally use an `assets` path
  let handle = asset_server.load("https://raw.githubusercontent.com/bevyengine/bevy/refs/heads/main/assets/branding/bevy_bird_dark.png");
  commands.spawn(Sprite::from_image(handle));

Further Work

  • Better AssetReaderError: The asset path is not always collected in the error. This is likely a common error that actual non-developer users will see so its important to get it right.
  • Possibly expose the native request mechanic as a general fetch, in the future we'll have many reasons to be making requests.

@mrchantey mrchantey added C-Feature A new feature, making something new possible A-Assets Load files from disk to use for things like images, models, and sounds C-Dependencies A change to the crates that Bevy depends on S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it X-Controversial There is active debate or serious implications around merging this PR M-Needs-Release-Note Work that should be called out in the blog due to impact D-Modest A "normal" level of difficulty; suitable for simple features or challenging fixes labels Feb 17, 2025
@alice-i-cecile alice-i-cecile added S-Needs-SME Decision or review from an SME is required and removed S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it labels Feb 17, 2025
@Elabajaba
Copy link
Contributor

Just a quick note, ring is now unmaintained for the foreseeable future.

@TimJentzsch TimJentzsch added the S-Needs-Review Needs reviewer attention (from anyone!) to move forward label Feb 25, 2025
@recatek
Copy link
Contributor

recatek commented Mar 5, 2025

Adding MPL as an allowed license seems like it would have complications for distributing game binaries, no?

That said, I'm not clear what new dependency here actually requires it. Both ureq and webpki-roots support MIT and Apache 2.0 from looking at their repos.

@mockersf
Copy link
Member

mockersf commented Mar 5, 2025

Just a quick note, ring is now unmaintained for the foreseeable future.

ring is maintained again, now by the rustls team

@mockersf
Copy link
Member

mockersf commented Mar 5, 2025

blocked on finding where the MPL license comes from and removing it

@mockersf mockersf added the S-Blocked This cannot move forward until something else changes label Mar 5, 2025
@mockersf
Copy link
Member

mockersf commented Mar 5, 2025

waiting on a new release of webpki-root with this commit: rustls/webpki-roots@90c48f3#diff-8aec0a2c31031e00524df726f854726660ec8289dcddfa70005000fc21b00511

@jf908
Copy link
Contributor

jf908 commented Mar 5, 2025

As of right now, ring is not actually included in this PR, it was replaced by aws-lc-rs. Now that ring has removed the OpenSSL parts of the license, I think we can remove aws-lc-rs, remove OpenSSL from deny.toml and use the ureq's default ring dependency again. This would be especially nice because aws-lc-rs requires installing CMake on Windows.

@alice-i-cecile alice-i-cecile added S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged and removed S-Blocked This cannot move forward until something else changes S-Needs-Review Needs reviewer attention (from anyone!) to move forward S-Needs-SME Decision or review from an SME is required labels Mar 6, 2025
@mrchantey
Copy link
Contributor Author

I agree, more iteration is requried but this pr is already substantial. How about we create a follow-up issue for discussion on whats needed to complete http asset sources and open smaller prs from there.

@mrchantey mrchantey removed the S-Blocked This cannot move forward until something else changes label May 19, 2025
@mrchantey mrchantey requested a review from mockersf May 19, 2025 00:27
@alice-i-cecile alice-i-cecile added the S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged label May 19, 2025
@mrchantey mrchantey removed the S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged label May 20, 2025
@alice-i-cecile alice-i-cecile added S-Needs-Review Needs reviewer attention (from anyone!) to move forward X-Blessed Has a large architectural impact or tradeoffs, but the design has been endorsed by decision makers and removed X-Controversial There is active debate or serious implications around merging this PR labels May 21, 2025
@alice-i-cecile
Copy link
Member

I've spoken with @cart about this, and we'd like to move forward with this as a solution for example assets (in addition to thinking it's broadly useful). We just need a second approval now to get this merged.

@alice-i-cecile alice-i-cecile added this to the 0.17 milestone May 21, 2025
alice-i-cecile and others added 2 commits May 22, 2025 11:52
Co-authored-by: jf908 <jf908@users.noreply.github.com>
Copy link
Member

@mockersf mockersf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar to how the file asset source exposes only a directory and not the whole filesystem, I think an https source should only allow access to a domain and not the whole internet

Copy link
Contributor

You added a new feature but didn't update the readme. Please run cargo run -p build-templated-pages -- update features to update it, and commit the file change.

mockersf and others added 5 commits June 29, 2025 00:52
* Rename and split http_source feature to http and https
* Enable http sources when using processed asset mode
* Fix ureq/rustls feature gate (it was only working by accident because ureq with full features is part of the dev dependencies)
* Disable gzip and json ureq features
* Misc. rewording of a couple docs
* Setup HttpSourcePlugin as part of DefaultPlugins to allow for future configuration (there is currently none)
@jf908
Copy link
Contributor

jf908 commented Jun 30, 2025

similar to how the file asset source exposes only a directory and not the whole filesystem, I think an https source should only allow access to a domain and not the whole internet

I am quite biased towards allowing more than 1 origin because I’m currently making a bevy game where not all my assets are hosted over the same server. Otherwise I see the upside of being able to neatly switch out the asset server host name but there’s also the downside that we would lose the neatness of the asset path being the URL.

Do you think a config option to specify allowed host names would be a good alternative?

@jf908
Copy link
Contributor

jf908 commented Jul 1, 2025

I have given this some more thought and I'm starting to turn around to the idea.

I don't think a config option for allowed host names would be that useful since its quite easy to parse the url and do any checking you want manually before you load an asset.

I didn't know this until now but file assets can be loaded from absolute paths but they will be rejected unless unapproved_path_mode is set accordingly. We could probably do the same for web assets by having the same unapproved_path_mode option to allow access to the entire internet if required. The main problem I see is that relative paths already default to the file system and afaik there is no common syntax to define a relative http URL so we would have to come up with our own syntax which doesn't clash with absolute http URLs which I don't think would be very intuitive.

Another issue is that unlike the assets folder, there is no sensible default origin. If we want to force people to pick an origin (which would be good practice), we wouldn't be able to make the Plugin Default and that would make it annoying to construct (in the future when it has many config options).

@mockersf
Copy link
Member

mockersf commented Jul 1, 2025

If we want to force people to pick an origin (which would be good practice), we wouldn't be able to make the Plugin Default and that would make it annoying to construct (in the future when it has many config options).

I think this is more a feature than an issue. Having to opt in to http on a specific domain is a better way to use it in my opinion. If the number of config options becomes too large, we can either rely on a default implementation for most options, or a builder pattern

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Assets Load files from disk to use for things like images, models, and sounds C-Dependencies A change to the crates that Bevy depends on C-Feature A new feature, making something new possible D-Modest A "normal" level of difficulty; suitable for simple features or challenging fixes M-Needs-Release-Note Work that should be called out in the blog due to impact S-Needs-Review Needs reviewer attention (from anyone!) to move forward X-Blessed Has a large architectural impact or tradeoffs, but the design has been endorsed by decision makers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature: Upstream bevy_web_asset, allowing assets loaded via http
8 participants