Organizing Sidecar assets

[jonspalmer]

Introduction

There has been a lot of discussion on the right way to organize sidecar assets and ruby classes for ViewComponents: #15 (comment), #67, #168, #283.

We don't have a clean solution yet but we have some ideas

What do we know?

1. ViewComponents are more than just a subclass of ViewComponent::Base and a view file. It's clear that the community has spoken for a need for sidecar javascript assets - Stimulus controllers, css, i18n yml, and also other Ruby files like Stimulus Reflex.
2. The current directory structure is flat and leads to very fat directories
3. Namespaces help but don't solve the complete problem - they just subdivide it.
4. Ruby autoloader rules impose restrictions on how we organize things.

The Problem today

Suppose we have four components: AlertComponent, ButtonComponent, FormComponent and Form:TextFieldComponent (note the Ruby namespace on that later - it's going to matter). Today the directory structure would look like this:

app/components

├── alert.css
├── alert.html.erb
├── alert_component.rb.          => AlertComponent
├── alert_controller.js
├── application_component.rb     => ApplicationComponent
├── button.css
├── button.html.erb
├── button_component.rb.         => ButtonComponent
├── button_controller.js  
├── form
│   ├── text_field.css
│   ├── text_field.html.erb
│   ├── text_field_component.rb  => Form::TextFieldComponent
│   └── text_field_controller.js
├── form.css
├── form.html.erb
├── form_component.rb            => Form::FormComponent
└── form_controller.js

We can see app/components is already a 'fat' directory and we're only four components in.

What would the ideal solution look like?

In an ideal world the solution would look much like organizing front end assets where we group each component and its sidecar file in their own directory:

app/components
├── alert
│   ├── alert.css
│   ├── alert.html.erb
│   ├── alert_component.rb
│   └── alert_controller.js
├── application_component.rb
├── button
│   ├── button.css
│   ├── button.html.erb
│   ├── button_component.rb
│   └── button_controller.js
└── form
    ├── form.css
    ├── form.html.erb
    ├── form_component.rb
    ├── form_controller.js
    └── text_field
        ├── text_field.css
        ├── text_field.html.erb
        ├── text_field_component.rb
        └── text_field_controller.js

Solutions

1. Use Autoloader

At first glance we can use the auto loader to tell Rails to find our components inside directories (much like Rails eager loads ruby classes in all sub-directories of app

Rails.application.config.paths.add "app/components", eager_load: true, glob: "*"

This works great except for namespaces. AlertComponent, ButtonComponent, FormComponent work exactly the way we work but TextFieldComponent won't get loaded at all.

Also add namespaces to the autoloader - nearly works 😄 😞

We can add the namespaced components to the autoloader

Rails.application.config.paths.add "app/components", eager_load: true, glob: "*"
Rails.application.config.paths.add "app/components/form", eager_load: true, glob: "*"

Now Rails will load the TextFieldComponent However it will lose it's 'Form' names space so it will be TextFieldComponent not Form::TextFieldComponent

There are other combinations of autoloader configuration, particularly with Zeitwerk in Rails 6 that allow us some flexibility. For a detailed discussion see fxn/zeitwerk#111. However, those solutions require very careful inclusion and exclusion of directories from the autoloader. It's not clear that we could automate the process to detect namespaces etc.

Pros:

• Gives exactly the directory structure we want

Cons:

• Complicated autoloader configuration
• Non-standard Ruby/Rails file layout (will IDEs get this right?)
• Hard to automate
• Probably needs manual configuration when introducing new namespaces/components

2. Sidecar assets in a component directory

Leave the Ruby code as is but put the sidecar assets in a component specific directory.

app/components
├── alert
│   ├── alert.css
│   ├── alert.html.erb
│   └── alert_controller.js
├── alert_component.rb
├── application_component.rb
├── button
│   ├── button.css
│   ├── button.html.erb
│   └── button_controller.js
├── button_component.rb
...

The code we'd need here would be to change the component view resolution paths to look in the appropriate sub directory. js, css likely just work - would need to check the details of parent-child Stimulus controllers etc.

Pros:

• Standard Rails autoloading
• Gets the sidecar files 'close' to the components

Cons:

• Component Ruby files outside of "their" directory
• Complicates the component view loading logic - perhaps per component
• Doesn't solve for Ruby sidecar files like StimulusReflex

3. Sidecar assets in one assets directory

Leave the Ruby code as is but put the sidecar assets in an assets directory with sub directories that match the components directories.

app/components
├── alert_component.rb
├── application_component.rb
├── button_component.rb
├── form
│   └── text_field_component.rb 
├── form_component.rb
└──assets
    ├── alert
    │   ├── alert.css
    │   ├── alert.html.erb
    │   └── alert_controller.js
    ├── button
    │   ├── button.css
    │   ├── button.html.erb
    │   └── button_controller.js
    └── form
        ├── form.css
        ├── form.html.erb
        ├── form_controller.js
        └── text_field
            ├── text_field.css
            ├── text_field.html.erb
            └── text_field_controller.js

...

The code we'd need here would be a single change the component view resolution paths to look for views in the assets directory. js, css likely just work.

Pros:

• Standard Rails autoloading
• Single point change in ViewComponent for view resolution
• Similar to Rails patterns for controller/views (though you can argue this is a Con too 😉)

Cons:

• Views and assets are much further away from their components
• Complicates the component view loading logic
• Doesn't solve for Ruby sidecar files like StimulusReflex

Proposal

I'd lean towards a configuration option to enable 2 unless the actual implementation is sufficiently complicated that 3 looks more appropriate.

I'd love for 1 to work after a lot of experimentation I can't see a clean way to bundle it as part of the gem that covers all reasonable use cases without a lot of manual configuration.

[jonspalmer]

ping @jaredcwhite, @rosendi, @nicolas-brousse, @joelhawksley, @joelmoss

Would love to hear your comments and thoughts given you've been deep in the prior conversations and likely have real world experience and thoughts of the pros & cons.

[joelmoss]

Solution 2 is pretty much exactly what I've been using for over a year now in a current production app, and it works nicely. Never had any issues, and it's clear where everything is.

I'm not actually using ViewComponent, but looking to integrate it, so I obviously want to keep the structure I already have.

One difference between this and my current setup, is the naming of the assets. You propose naming these identical to the component name, but I use index. eg. app/components/button/index.html.erb and app/components/button/index.css.

Index is obviously a well known convention, and avoids duplicating the name everywhere. I also find it easier to identify other files that may exist in the same directory, such as those which are imported (JS), or other related files.

[nicolas-brousse]

Hi @jonspalmer

Solution 2 sounds the better for me, with the current state of this gem and Rails autoloading.
Also, it struture is simple enough if we need to migrate from 2 to 1. If we found a way to achive it in the future.

Solutions

1. Use Autoloader

After having working on this several hours too, I confirm that it's not clean enough as solution to be considered. At least for now.

2. Sidecar assets in a component directory

The last time I've worked on structuring components with view_component, I've considered this structure too. Sounds as a good compromise IMO.

About cons:

• Component Ruby files outside of "their" directory

I guess it's a small cons.

• Complicates the component view loading logic - perhaps per component

Does it complicate it that much?

• Doesn't solve for Ruby sidecar files like StimulusReflex

I think I've missed some conversation about this. What is expected with StimulusReflex?

3. Sidecar assets in one assets directory

About cons:

• Views and assets are much further away from their components

When we work in controllers and views, I think we are not often working on both files at the same time. But for the components it may be the case for components, since in components we work in a smaller piece.

• Complicates the component view loading logic

Same question that for solution 2.

• Doesn't solve for Ruby sidecar files like StimulusReflex

Same question that for solution 2.

[danturu]

I'm currently refactoring all the components I have in my new app. I will post my thoughts in a week or so.

[jonspalmer]

I threw up a quick version of option 2 in #335.

There are some questions around supporting partials but the code seems reasonable and not too invasive.

[mickenorlen]

I would give my arm for option nr 1, it just looks so amazing. Mb it could still be left open as an option for experimental people to try in rails 6?

[jaredcwhite]

Hi all, sorry I'm a bit late jumping into the conversation here. I totally understand the issues surrounding getting Option 1 to work…yet I feel that's exactly what a lot of people will be looking for, especially if they're coming at this from other contexts such as React dev. I'd personally be comfortable with Option 2 as a quicker path forward for now, but I think dropping Option 1 for an indeterminate period of time is unwise.

Also, from a "Storybook.js" type perspective, even stories/tests live in the same component folder. I'm not necessarily advocating for that, but it's just another example of the inertia to keep everything about each component within a single folder.

[jonspalmer]

@mickenorlen & @jaredcwhite I'm in total agreement that Option 1 is what we want. Sadly I just don't think Ruby/Rails supports it in any reasonable way. If you never want to use namespaces it can be made to work great. Whithout namespaces (or subfolders) you don't need that many components before things become unmanageable.

Perhaps there is some clever combination of feature requests we can put in for Zeitwerk that would give us the pattern that we want but the conversation in fxn/zeitwerk#111 suggests is unlikely to work "out of the box"

[mickenorlen]

Maybe something like this could be an option?

• Folder structure like @rosendi's How do you organize components? #67 (comment) - for maximized searchability of main component files.
  But as the idea presented itself in my mind now I'm also naming the directory with trailing _component.

components/my_component/my_component.rb

• Zeitwerk loading as is, giving us the long namespace MyComponent::MyComponent.

• But then we give users the option to delegate/redirect any calls from MyComponent to MyComponent::MyComponent I Imagine it's not too costly or difficult? Basically wherever there is a directory that ends with _component in components/. That folders path relative to /components root could be classified and made into a delegator on initialization?

Or maybe it could be something like: if a class is not found and its name ends with Component. Make another check for the nested Component? I duno...

• This way we would also be able to support multiple entrypoints/variants per component. For example a js version and a html version as discussed here: Rendering a component with both js.erb and html.erb #328 I'm sure this has many other use cases.

So then we could have something like

app/components
├── alert
...
└── form
    ├── form_component
        ├── form.css
        ├── form.html.erb
        ├── form_component.rb
        ├── form_controller.js
        └── text_field

    ├── text_field_component  
        ├── text_field.css
        ├── text_field.html.erb
        ├── text_field_component.rb
        └── text_field_controller.js

         # HERE is an example of alternative entrypoints for text_field_component!
        ├── js.rb 
        ├── js.js.erb
        ├── variation2.erb # And some other variation
        ├── variation2.html.erb

So now we could reach all of this as expected

# With some kind of aliasing/delegation
Form::FormComponent => Form::FormComponent::FormComponent
Form::TextFieldComponent => Form::TextFieldComponent::TextFieldComponent

Form::TextFieldComponent::Js # Will render js variant
Form::TextFIeldComponent::Variation2 # Another variant

• I realize in this specific example we would have to load not only _component.rb files, but all .rb files inside components.

Do you think it would be too "unrailsy" to support configuration options such as

  find_assets_in_same_folder = true/false
  implement_shorthand_delegation_namespace = true/false

Thanks!

[jonspalmer]

@mickenorlen Thanks for the thoughts. A couple of comments:

Or maybe it could be something like: if a class is not found and its name ends with Component. Make another check for the nested Component? I duno...

I think what you'r saying here is we iterate over the file system looking for classes etc. I'd though about that kind of pattern and come to the conclusion is super risky a) I worry about the performance of the iteration (maybe this doesn't matter) I've done things like this in a test suite (make sure every instance of a class has an instance of a matching spec) b) does it play nice in dev environments (ie when you don't compile everything up front (which is when you'd probably do the iteration) c) it's restrictive in the sense I can't put base classes where I want them. For example if you have an ApplicationComponent that everything inherits from its should live in app/components not app/components/application or ``app/components/application_component`. You get the same problem as you get into names spaces. Where does a base class for all "Form" Components live? It shouldn't have sidecar view, js, css files etc. I worry the presence of files like that break the directory iteration strategy.

To your idea of extra nesting I like it and it feels easy enough to do. My sense is you'd organize it like this:

app/components
├── alert
...
└── form
    ├── form_component
        ├── form.css
        ├── index.html.erb
        ├── component.rb
        └── form_controller.js
    ├── text_field_component  
        ├── text_field.css
        ├── index.html.erb
        ├── component.rb
        └── text_field_controller.js

Note I don't think we need full delegation I think we need some syntactic sugar to add a new method to the module so that

Form::FormComponent.new => Form::FormComponent::Component.new

I think there are a few potential gotchas
a) its just a bit weird :)
b) I worry about the greppability and IDE support. When you have Form::FormComponent.new in a view are you going to "find" that it's really defined in form/form_component/component.rb ?
c) does this layout play well with Stimulus naming conventions. See @jaredcwhite's #168 (comment)

I'm curious about your thoughts around variants and formats. I'd expect to pass a format option to new rather than there being completely new Ruby component classes. Perhaps we can continue that conversation in #328

[juanmanuelramallo]

Is anyone familiar on how React projects organize their component-related assets?

According to the docs https://reactjs.org/docs/faq-styling.html it seems like they don't impose anything, in fact they state:

React does not have an opinion about how styles are defined;

I think that goes for any kind of assets (images, even translations).

Given real projects might be different than what the documentation says (#67 (comment)), and the amount of tooling around transpilation and bundling in the JS land, what I wanted to bring to the discussion here is:

Do we really want ViewComponent to have a word on this topic? Why not leave the decision to each project/devs?

I think we can just provide a section in the readme for some suggestions (like we already do here https://github.com/github/view_component#sidecar-assets-experimental) (and also as React does in their docs).

[jaredcwhite]

Well it's a little different over in JS world because you have to import every code dependency explicitly in every file. That often expands to stylesheets, images, etc. as part of the Webpack build process. So you'll see things like:

// ./Button.jsx
import "./Button.css"
import "./Icons.svg"
…

Because instead we're autoloading everything in helpful Ruby world, it means the hard work of importing proper dependencies falls on the tooling not the end-user developer. (As it should be IMHO!)

[mickenorlen]

@jonspalmer,

(i said) Or maybe it could be something like: if a class is not found and its name ends with Component. Make another check for the nested Component? I duno...

I think what you'r saying here is we iterate over the file system looking for classes etc

I was thinking more if it's possible to hook into the "class not found functionality" to prepend an additional clause to check another class before raising the error. This code would then only be run if the class that is missing ends with Component (and maybe something like if the corresponding file that defined the class is to be found in the /components directory). But as you mentioned IDE lookups would probably be difficult. I guess that works out of the box with unconventional zeitwerk loading patterns?

Where does a base class for all "Form" Components live?

As we've separated namespace folders from component folders, the form namespace is open for any file you like - as long is it doesn't clash with your component names. Couldn't something like this work then?

└── form
    ├── base_component.rb
    ├── another_base_component.rb    
    ├── form_component
        ├── another_base_in_here_if_needed.rb
        ├── index.css
        ├── index.html.erb
        ├── index.rb
        └── index_controller.js

But I understand your points that delegation can be problematic. It seems a potential idea to delegate in the new-call instead. If we could make it work it could be a good option but otherwise I'm thinking delegation may not be the most important thing in the end. I could still be happy simply giving a "name" to the main component namespace, such as index/main/html (example below). I think the key for me is to add that next level of nesting so the directory itself defines the component namespace.

 app/components
├── alert
...
└── form
    ├── form_component
        ├── index.css
        ├── index.html.erb
        ├── index.rb
        └── index_controller.js
    ├── text_field_component  
        ├── index.css
        ├── index.html.erb
        ├── index.rb
        └── index_controller.js

Form::FormComponent::Index # works decently :)

Regarding the component variants (#328). I know it's a discussion in itself and I don't know which way is best. For me it just seems to make sense to keep all options open, also then i would be up and running with multiple variants today without having to add on multi template file/format management features and conditionals to the component base/render?/initialize methods etc. I just inherit from index or some base and tweak what i need.

Also agree with @juanmanuelramallo that the user could be left more options to decide how to structure these files, within reason of course :)

[jonspalmer]

Personally I'm not at all a fan of Form::FormComponent::Index.new. The Index has no meaning we've introduced another noun that the end user of the component shouldn't need to think about.

In terms of variants I hear you on the flexibility. However, I think the problem breaks down to a) if all that is changing is a different way to render (a view or a format) then it should be one class/component and multiple view files. Creating new trivial Ruby component classes just to get a new view file seems sub-optimal b) if the logic changes for the 'variant' (ie different Ruby code) then it's just a new component and you're already free to use inheritance or composition to get what you want.

[mickenorlen]

Yeah, i guess variants usually wouldn't need new component files.. I haven't used this gem long enough to say for sure. But the very simple js and html example i made in #328 seems to need more or less another instance variable (where to inject the html) a different helper method (the html to inject instead of js to inject to the component template), maybe a slightly altered render?/validation of input variables... I don't know, I just had this feeling that more options = good

[mickenorlen]

Another (possibly strange) idea.

When you generate a new component form/form_component. You get this conventional structure below. But imagine that form_component.rb is NOT the end component. instead it functions kind of like a configurable entrypoint/controller/base (that you usually don't need to edit) that in turn delegates to index.rb or whatever you want to call the end component.

app/components
...
└── form
    ├── form_component.rb
    ├── form_component  
        ├── index.css
        ├── index.html.erb
        ├── index.rb
        └── index.js

Then there could be two rather clear cut options for users.

1. Put the end component straight in form_component.rb and get assets in separate folder.
2. Or use the new special delegator class to enable multiple variants/entrypoints etc and keeping all (most relevant) assets in same folder.

Then you could even decide to have different structures between different components to suit your needs as the special delegator class would handle the configuration for you when needed. And it could take options and inherit from an ApplicationComponentDelegator so you could set it up just the way you like with expected naming etc

[joelmoss]

Oh now I really like this idea @mickenorlen

[jonspalmer]

I'm not sure I understand the nuances of this idea. What would be the example of the Ruby code going in form_component.rb vs index.rb?

[mickenorlen]

index.rb would be the regular view_component. (It could be named component.rb or whatever)

form_component.rb is basically the delegation we've been talking about in it's own dedicated auto generated file (no more magic!). It could then function sort of like a controller and delegate any calls from Form::FormComponent to the Form::FormComponent::Index (the user configured target namespace). Maybe it could also function as a base class IF multiple component variants would become a thing. Not sure what more you could use it for... apart from the configuration of how you want to structure your files :D

[mickenorlen]

Woah! Look at this simple class alias implementation:
https://stackoverflow.com/questions/7795809/class-alias-in-ruby

If I understand things correctly this could give us another option:

class Form::FormComponent::Index < ApplicationComponent
  #...
end

Form::FormComponent = Form::FormComponent::Index

With wanted structure

app/components
...
└── form
    ├── form_component  
        ├── index.rb
        ├── index.css
        ├── index.html.erb

And

Form::FormComponent.new

[mickenorlen]

Haven't gotten it to work... ActionView::Template::Error undefined method 'new' for Form::FormComponent:Module.. guessing our component directory leads to creation of a module which is overwriting anything i write there..

[mickenorlen]

Got it to work with config.to_prepare loading which I've been using to override functionality in engines. Not sure if it could be an option? :S
https://edgeguides.rubyonrails.org/engines.html#overriding-models-and-controllers

application.rb

config.to_prepare do
  Dir.glob(Rails.root.join('app', 'components', '**', '*.rb')) do |f|
    require_dependency(f)
  end
end

It didn't seem to work with aliasing the whole namespace as that broke the path to Index.
uninitialized constant Form::FormComponent::Index::Index

But now i can modify the Form::FormComponent module

app/components/form/form_component/index.rb

class Form::FormComponent::Index < ApplicationComponent
  #...
end


Form::FormComponent.instance_eval do
  def new(*args)
    Form::FormComponent::Index.new(*args)
  end
end

Hmm. Could this be something? :P again, thanks for taking the time!

[mickenorlen]

Possible simplification, allows any naming of the nested main component file.

class ComponentHelper
  def self.set_main
    call_file = caller[0][/[^:]+/]
    component_path = File.dirname(call_file)
    component_rel_path = Pathname.new( component_path ).relative_path_from( Pathname.new( Rails.root.join('app', 'components') ) ).to_s
    component_namespace = component_rel_path.classify
    component_name = File.basename(call_file, ".rb").classify

    component_namespace.constantize.define_singleton_method('new') do |*args|
      "#{component_namespace}::#{component_name}".constantize.new(*args)
    end
  end
end

class Form::FormComponent::Index < ApplicationComponent
  #...
end

ComponentHelper.set_main

[joelhawksley]

🤯 WOW!!! Thanks for the thorough, thoughtful discussion folks! This is fantastic.

Given all that has been said here, I think we should go with:

2. Sidecar assets in a component directory. Leave the Ruby code as is but put the sidecar assets in a component specific directory.

As it is an incremental change that doesn’t require any changes to the way we’re loading Ruby files. We can build on this approach as time goes on.

@jonspalmer how about we start there and see how it feels? I know you have a WIP PR for that approach, so let’s see about getting it ready to merge!

[danturu]

2. Sidecar assets in a component directory. Leave the Ruby code as is but put the sidecar assets in a component specific directory.

Most of components (I would say 90%) have only .css and .html in addition to .rb. Don't you think it won't be a big win?

There is a fourth option leave it as is and use modules for grouping components alongside with the flatten folder structure:

PullRequests::ItemComponent
PullRequests::ListComponent
PullRequests::TimelineComponent

...
└── pull_requests
        ├── item_component.rb
        ├── item.css
        ├── item.html.erb
        ├── list_component.rb
        ├── list.html.erb
        ├── list.css
        ├── timeline_component.rb
        ├── timeline.html.erb
        ├── timeline.css

Later if pull_requests grows (for example because of complexity of Timeline) it's time to break it into a few modules again or create a new submodule:

PullRequests::ItemComponent
PullRequests::Timelines::ConnectorComponent
PullRequests::Timelines::DiscussionCommentComponent

...
└── pull_requests
        ├── item_component.rb
        ├── item.css
        ├── item.html.erb
        └── timelines
                  ├── discussion_comment_component.rb
                  ├── discussion_comment.html
                  ├── discussion_comment.css
                  ├── connector_component.rb
                  ├── connector_component.html
                  ├── connector_component.css

A few month ago I started a new project with 1 option (since it feels like a React way). Later I flattened my components and break them into logical peaces using modules and it feels OK. The only problem is with the UI library because the UI's components can't be grouped somehow:

(self-packed)

(flattened)

I don't think the second option could help here either.

[joelhawksley]

@rosendi I totally agree! For most cases, modules should do the trick: it's the approach we're taking here at GitHub.

The subfolder architecture will be optional, and benefit folks using sidecar CSS and JS, in my opinion.

[jonspalmer]

That's for all the great discussion. As a first step we're moving along with a simple implementation of option 2. See #335.

Overall we're trying to balance a few things:

• Staying true to Principal of Least Surprise
• Directories have meaning in Ruby - they're namespaces. We need to stay true to this principal.
• Where possible we're like to keep this library compact, simple and Rails like. There is still a desire to upstream this project into Rails in some future version. If we drift too far we make that less likely.
• Provide flexibility, via code hooks if necessary, for consumers to extend and configure ViewComponents to their liking (you can do a lot by defining your own 'magic' in your ApplicationComponent)

I'm curious to the last point if there is a way we could provide a configuration mechanism or expose a method to define your own "view resolution" so consumers to use whatever scheme they like. I have't spent much time thinking about it but if others wanted to we'd be happy to consider it.

[joelhawksley]

Thank you everyone for your thoughtful contributions to this discussion! I'm really happy with the solution we've landed:

#335

Thank you to @jonspalmer for taking this on and guiding us to a simple, flexible solution.