Goldiloader for Arrays

[apauly]

There might be situations where it's not possible to use a simple ActiveRecord::Relation. So, I was wondering if there is already a way to add the auto_include_context logic to a plain array of records?

Correct me if I'm wrong, but I didn't find any obvious out-of-the-box solution for this scenario.

I was thinking about sth like this:

records = Goldiloader::Collection.new([record_a])
records << record_b
records << record_c

records.map { _1.auto_include_context.models.count }
=> [3, 3, 3]

Would you be interested in a PR adding that functionality?

[jturkel]

@apauly - Can you elaborate on the use case this would solve? Is it automatic eager loading of associations for models that have been loaded through separate queries or perhaps only a subset of models loaded by a query?

[apauly]

@apauly - Can you elaborate on the use case this would solve? Is it automatic eager loading of associations for models that have been loaded through separate queries or perhaps only a subset of models loaded by a query?

I am currently looking at two use cases:

• List of AR objects wihtout a relation
  Our use case is a complex nested form. Setting up the records for a specific nested relation includes custom logic and mixes existing records and new records. It's currently implemented by filling an array that is passed to fields_for. As it's not a AR relation, the auto_include_context for each record will only include the record itself.
  Instead of using a plain array, we could use a Goldiloader::Collection instead and benefit from preloaded relations and custom preloads.

• Reporting/Metrics/BI
  The second use case is about custom SQL queries that we have for reporting purposes. Including aggregations, grouping, etc - in the end, it's an array of hashes.
  As we don't have any AR objects involved here, there are obviously no relations that could be loaded by goldiloader automatically. However, if we'd use a Goldiloader::Collection, this would still allow us to use custom POROs with custom preloads to save a lot of queries.

One big motivation for trying to utilize goldiloader in both of these situation is the fact that we're dealing with a lot of existing code that includes indirection. Something like a Goldiloader::Collection would allow us to fix a lot of N+1 queries while defering some refactoring decisions to a point when we have more confidence about how to do it.

[jturkel]

Sounds like some great use cases. I'd definitely be open to a PR assuming it didn't add too much complexity to the gem. A few thoughts on the API:

• The first use case is already possible using the private Goldiloader::AutoIncludeContext.register_models(models) API. Can we expose a similar public API on Goldiloader?
• For the PORO use case, would we expose a module that needs to be included with the necessary context tracking and custom preload hooks?

[djezzzl]

Hi @apauly,

I might be missing something, but I think you could use N1Loader for that, given its custom-isolated-loaders functionality. Of course, mixing with goldiloader has become possible, so you don't need to worry about context setting.

https://github.com/djezzzl/n1_loader