When running assets:precompile, Propshaft includes individual digested files for all CSS / JS instead of only bundles

[nickjj]

Hello,

I'm using Rails 8 with Propshaft v1.1.0 along with jsbundling and cssbundling where esbuild is doing the bundling.

After running assets:precompile in production mode the following files are created in public/:

The concern here is all of the JS and CSS files. When I was using Sprockets, only the application-xxx.js and application-xxx.css files were created but with Propshaft it's generating digested files for everything.

Based on the Propshaft documentation I thought maybe since bundling is happening with esbuild I could set:

Rails.application.config.assets.excluded_paths << ["app/assets/stylesheets", "app/javascript"]

But this resulted in the same outcome where the excess JS and CSS files were created.

What am I doing wrong?

Edit: this is happening with Sprockets too

Here's the same project but using Sprockets:

Is this possibly a Rails 8 change and has nothing to do with Propshaft? Unless I'm crazy (which is very possible), I don't recall all of these files being created in the past by Sprockets. Do we know why they're being created?

[midnight-wonderer]

I spotted three question marks, I could only help with some.

The digest is intended to be used with Cache-Control: immutable. I hope you already know this, I just provided the answer as feedback to your question.

Assuming that the inverse of this statement is what you want:

Propshaft includes individual digested files for all CSS / JS instead of only bundles

Have you read bypassing the digest step yet?
It is basically Propshaft's interface for working with other build tools (esbuild included); can it solve your problem?
Why or why not?

[theodorton]

The reason your excluded_paths config doesn't work is because you're not setting absolute paths. You'll need to use Rails.root.join. From the README:

You can however exempt directories that have been added through the config.assets.excluded_paths. This is useful if you're for example using app/assets/stylesheets exclusively as a set of inputs to a compiler like Dart Sass for Rails, and you don't want these input files to be part of the load path. (Remember you need to add full paths, like Rails.root.join("app/assets/stylesheets")).

[nickjj]

The reason your excluded_paths config doesn't work is because you're not setting absolute paths.

Thanks, I missed that but I set that and it's still digesting everything in stylesheets and javascript dependencies.

For example it still produced this output:

The project I'm using doesn't even use Trix so I don't know where that's coming from, but that's a separate issue.

That output was with config.assets.excluded_paths += [Rails.root.join("app/assets/stylesheets"), Rails.root.join("app/javascript")].

The result I'm looking to get is:

• I have all of my assets in the default CSS and JS locations with Rails using Tailwind and vanilla JS
  • This is working
• I am using esbuild to bundle that
  • This is working
• I want Propshaft to only produce application-abc123.js and application-abc123.css digests based on the bundles esbuild created instead of producing a bunch of individually digested CSS / JS files
  • This is not working
• I still want Propshaft to perform digest replacements in CSS / JS assets (such as url and RAILS_ASSET_URL("..."))
  • This is not working

You can reproduce this in my Rails starter app project at: https://github.com/nickjj/docker-rails-example

To run it in production mode you'd modify the .env file and set these:

export COMPOSE_PROFILES=postgres,redis,web,worker,cable
export RAILS_ENV=production
export NODE_ENV=production
export DOCKER_WEB_VOLUME=./public:/app/public

Then docker compose up --build and check the public/ directory.

You can run ./run clean to clean out those assets and other temp files in between tests.

[theodorton]

The project I'm using doesn't even use Trix so I don't know where that's coming from, but that's a separate issue.

Most likely happening because you're loading actiontext (as part of rails/all). You can pick individual libraries to avoid this.

I want Propshaft to only produce application-abc123.js and application-abc123.css digests based on the bundles esbuild created instead of producing a bunch of individually digested CSS / JS files

The rationale for including all assets when precompiling is to support modern nobuild techniques that don't require you to use CSS and JS bundlers. This is one of the biggest changes from sprockets and I doubt it will change in the short term.

You can run bin/rails assets:reveal:full to get a full list of assets found in propshafts load path. For anything sourced from a gem you have three options:

1. Comment out the require statement in application.rb to skip it (if it's part of Rails)
2. Remove the gem
3. Ignore the path

I still want Propshaft to perform digest replacements in CSS / JS assets (such as url and RAILS_ASSET_URL("..."))

This most likely happens because app/javascripts has been taken out of the load path. Propshaft must have access to dependencies in order to compute their digests as the files referring to them will get a digest that includes its dependencies. I'm not sure how well that works for already digested/built assets.

If you don't exclude app/javascripts and it's still not working, I would suggest that issue is reported separately.

---

Your main concern in this issue (that all files are included) is the expected behaviour at the moment, and there's an existing issue in #89.

[nickjj]

Most likely happening because you're loading actiontext (as part of rails/all)

Yep that is the case here.

The rationale for including all assets when precompiling is to support modern nobuild techniques that don't require you to use CSS and JS bundlers. This is one of the biggest changes from sprockets and I doubt it will change in the short term.

It's a little surprising Propshaft doesn't act on the app/assets/builds directory. Unless I'm missing something, isn't the build directory the "final final" source of truth? In that sense, it wouldn't matter if import maps and nobuild are being used or esbuild because all that matters is what's in that directory, how the files got there has no impact.

If Propshaft's goal is to analyze CSS / JS files to do replacements and then digest the files, if it operated on app/assets/builds then nobuild or esbuild solutions would be the same end result and there wouldn't be extra individual files generated for folks who want to use esbuild?

[theodorton]

It's a little surprising Propshaft doesn't act on the app/assets/builds directory. Unless I'm missing something, isn't the build directory the "final final" source of truth? [...]

If Propshaft's goal is to analyze CSS / JS files to do replacements and then digest the files, if it operated on app/assets/builds then nobuild or esbuild solutions would be the same end result and there wouldn't be extra individual files generated for folks who want to use esbuild?

The builds directory is just a convention for where external bundlers should place their build output. To Propshaft, files in the builds directory are just as relevant as files located within other folders inside assets, except for cases where files have identical (undigested) names, where it will give priority to files located in the builds folder.

[nickjj]

Thanks, in that case there might be a bug around exclusions.

If the expected order of operations is:

• Ignore exclusions
• Check the builds directory
• Act on found files

Then if I ignore the app's css and js directories then the expectation is I would only have processed and digested bundled files in the public/assets directory but that's not the case. Even with the above, I had a bunch of digested separate js / css files (including Trix which is loaded from Rails itself) in public/assets.

My previous comment has an example project with repeatable steps to verify that.

[theodorton]

Then if I ignore the app's css and js directories then the expectation is I would only have processed and digested bundled files in the public/assets directory but that's not the case. Even with the above, I had a bunch of digested separate js / css files (including Trix which is loaded from Rails itself) in public/assets.

Libraries that integrate with Rails (through railties) or Rails engines will add more paths to the load path if necessary. You'll need to pick the libraries you want to use (in config/application.rb) or add the paths added by the engine to config.assets.excluded_paths (see #89). This comment seems like a good workaround for your use case, but you won't be able to use the RAILS_ASSET_URL macro with that approach. Alternatively you can, as part of your deployment process, remove all javascript files in public/assets that you don't need.

[nickjj]

Libraries that integrate with Rails (through railties) or Rails engines will add more paths to the load path if necessary

The files being picked up by Propshaft are independent of that. These are CSS and JS files in my project that it's digesting, which were excluded.

but you won't be able to use the RAILS_ASSET_URL macro with that approach

This also seems like a bug?

If the builds directory is capable of being digested by Propshaft, then it doing a find and replace on those files to do the in-file CSS and JS digests should be included too?

Alternatively you can, as part of your deployment process, remove all javascript files in public/assets that you don't need.

That could work but adding a custom post-deploy step in all projects that use esbuild seems like it goes against the grain of convention over configuration?

[theodorton]

but you won't be able to use the RAILS_ASSET_URL macro with that approach

This also seems like a bug?

If the builds directory is capable of being digested by Propshaft, then it doing a find and replace on those files to do the in-file CSS and JS digests should be included too?

Sorry for not being precise. You won't be able to use RAILS_ASSET_URL if the assets you're referring to are not in the propshaft load path.

[theodorton]

Closing this. See workarounds and/or contribute to existing discussion in #89.