Hook mechanism for completion of Bundler.require

[logiclrd]

I'd like to add a mechanism to permit a Gem to register a block to be executed automatically once Bundler's Runtime has completed its require pass. I propose adding instance methods to Runtime and class methods to Bundler that would forward to the Bundler class's Runtime (Bundler.load) for this.

1. Code in a Gem would use syntax such as:

   Bundler.when_require_complete do
     # final initialization code here
   end
   

2. Runtime.when_require_complete would stash the provided block for later execution.

3. Runtime.require would, upon completion of its loop through the list of dependencies, call each registered block.

Thoughts that occur to me:

1. Does Bundler need to be threadsafe at all? What happens if two threads call require on the same Runtime object simultaneously? If that is a situation that currently works, then presumably I'll need to wrap accesses to the list of when_require_complete blocks in a Mutex.
2. If Bundler is called multiple times, of course the second invocation's require calls, if pointed at the same Ruby files, won't actually run any code. So, if we assume that the finalization code might need to run again if the loadout of Gems changes, the callback handlers need to persist from one call to Runtime.require to the next.

If this sort of functionality is likely to be approved by Bundler's maintainers, then I'll go ahead and make a pull request for them. :-)

Rationale:

We have a suite of Gems that tie into a common configuration subsystem, which is defined in its own Gem. Each Gem needs to add its embedded configuration directory to a central list before the configuration subsystem initializes itself and pulls in all of the values, based on order of precedence. Right now, we have no easy way to ensure that the configuration subsystem always initializes after all of the other Gems have had a chance to add their directories to the list. One approach we've tried is to place the variable initialization into the root file for the Gem that handles the configuration, and then arranging things so that it is the last Gem that Bundler.require brings in, but this has hit a snag with a Gem that requires a specific version (or newer) of the Gem that handles the configuration. Because it declares it as a dependency, that guarantees that the configuration Gem gets pulled in before this other Gem that wants to add to the configuration. We want to avoid having to explicitly initialize the configuration mechanism in every file that uses it -- there are hundreds of locations maintained by dozens of different teams.

Therefore, having the ability to register a block with Bundler to be automatically executed when the Runtime completes its require pass is a convenient mechanism to allow each of the Gems specified in the Gemfile (and brought in via dependencies) to initialize, and then use the configuration state data those initializations have built up to fire up the configuration subsystem.

[chrismo]

Could you code this up as a plugin?

[logiclrd]

I don't think so... There isn't currently any code at all in the Runtime.require method after the end of its loop through the dependencies. I could overwrite the function:

module Bundler
  class Runtime
    require_impl = instance_method(:require)

    def require(*group)
      require_impl.bind(self).(group)
      @hooks.each { |hook| hook.call() } unless @hooks.nil?
    end

    def when_require_complete(&block)
      @hooks ||= []
      @hooks << block
    end
  end
end

...but this would then depend on the specific internal implementation of Bundler. If the name of class Runtime, or the way that it is implemented, ever changed then the plugin would stop working.

Also... does Bundler actually have a plugin system that could handle this today?

[asutoshpalai]

This can be implemented as a hook plugin which is not present in bundle yet but is in the pipeline.

[logiclrd]

Oh? That's cool. :-) Is there some documentation of the ongoing development of hook plugins? I did some Googling, but could only find reference to plugins associated with bundle install and bundle update-type operations ("Lifecycle hooks"), not calls to Bundler.require from Ruby code.

[asutoshpalai]

Sorry, as this has not yet been implemented, so it's not documented.

[logiclrd]

How can it be in the pipeline, and yet have no planning, no discussion, no documentation and no code?

The reason I want to follow up on it is that, as mentioned, I've seen plenty of documentation of requests for plugin interfaces to automatically run code during bundle install or bundle update, but I haven't seen anyone looking for a "runtime" hook that would interact with the Bundle.require method itself. I wanted to make sure that what was in the pipeline actually covered that.

In issue rubygems/bundler#1945, @BMorearty asked about "before require" and "after require" hooks, which would obviously need to be in place before the call to Bundle.require starts. The thread did not address his questions, though.

In feature rubygems/bundler#8, plugins are much-discussed, but almost exclusively in the context of adding new commands to the bundle command-line interface. This appears to have actually been implemented in rubygems/bundler#3030.

In issue rubygems/bundler#3463, there is further discussion of plugins, but it specifically mentions only these types of integration:

• New subcommands, as implemented in Implemented a very basic plugin system. rubygems/bundler#3030.
• Source control plugins.
• Hooks registered, presumably as part of having a supporting Gem in the dependency tree, to run once a bundle install or bundle update completes.

Across all of this, @BMorearty's request is the only mention of the Bundler.require process, and even his would be something pre-registered. I'm looking for something that allows a Gem to tie into a post-all-requires event as it is being required.

Is that, specifically, in the pipeline?

Thanks :-)

[indirect]

Thank you for clarifying! That feature (to be clear: allowing a gem to add a callback once Bundler.require has completed, while it is being required by Bundler.require) is not in the pipeline. We haven't finalized the list of "lifecycle" callbacks, however, and before/each/after hooks for setup and require were at least on the list of possibilities. Thanks for this feedback, and we'll try to support this (or something else that can satisfy this use case) in the future.

[logiclrd]

Awesome, thanks very much :-)

[nestor-custodio]

How is this not a thing yet?? 🤔

[segiddins]

@nestor-custodio no one has sent a pull request with a tested implementation yet ;)