Provide a mechanism to auto-reload a Grape application

[lexer]

Grape app mounted to Rails is not reloaded on each request in development mode. Or may be I haven't configured it properly?

[mbleigh]

You'll find the same of Sinatra or any other Rack framework that you mount into a Rails app. Rails reloading is very specialized and doesn't "automatically" carry over to other frameworks. It's something we may consider in the future, but for now it's out. Sorry!

[mhenrixon]

This is really too bad because it makes it really frustrating to testin using guard and spork. How are people solving this today? Manually restart everything because spork/guard doesn't realize the api file has changed can't be very complicated in my scenario when it's just one file to reload but I can't make it happen.

Any suggestions to that? Going to the console and running rspec feels so 1999.

[mhenrixon]

@lexer you might want to have a look at mounting the app in config.ru instead of in the routes. That's how resque recommends mounting because of similar reasons and of course the assets reasons but I think it gives some benefits.

https://github.com/defunkt/resque/blob/master/examples/demo/config.ru

[mbleigh]

If you write your Grape app as an independent application you can have a separate set of tests just for it that will be quick. I have a Grape application with several hundred tests, no spork or anything like that, and it can load and execute the tests in less than 10 seconds.

[mhenrixon]

Sure but how do you share your rails app with that in a sane manner? I don't want to duplicate all the models and such and unfortunately writing the API after building the website.

Still haven't seen a good example of how to share all that code without some serious work.

I'd love to make the API truly independent because then I can host the API outside of the web app which gives a multitude of benefits but the pain of sharing the data access code is still putting me off.

[mhenrixon]

@mbleigh would you be using something like just activerecord outside rails like here http://blog.aizatto.com/2007/05/21/activerecord-without-rails/ ?

[dblock]

I am going to reopen this as a feature request. We do, in theory, want to provide a railtie to automatically reload a Grape API as well as a mechanism that works well with guard. It could be an external solution too. Any suggestions on how to accomplish this are welcome.

[mbleigh]

TL;DR I'm open to unobtrusive reloading, but I won't be working on it myself.

I guess it's true that we want reloading in an ideal world, but the "in practice" of this seems pretty unlikely. Especially now that there are mountable applications I can't really think of anything that can rebuild the application state short of reloading everything on every request, at which point we may as well not have reloading.

I also think that much of the impetus for wanting reloading is based on the use case of mounting Grape APIs inside of Rails. While I certainly want to support this use case, the reality is that Rails simply takes too long to boot and that's not Grape's fault. As I said, a bare Grape application can run hundreds of tests in about 10 seconds (with about 2 of those being boot time).

[mbleigh]

Haha, though now that I think about it since rebooting Grape is relatively fast, that's probably the solution right there. I don't know how you would do it, but completely blowing away all API classes and reloading them from scratch on each request would solve the Rails use case even if it would have no effect on bare apps.

[dblock]

I looked at implementing a Grape::API::reload!, very briefly. If something like that existed it would be trivial to add Rack::Reloader in front of it (which would reload individual files) and probably easy to trigger reload! if anything within the API were to change. That would work for Rails out of the box.

An alternative would be to look at how ActiveSupport does dependencies to trigger reloading, but either way we need something in Grape proper that dumps the existing API endpoint and recreates it from scratch.

For bare Rack applications that are rackup-ed you can use Guard with fresh-from-this-morning guard-rack gem.

[mhenrixon]

@mbleigh that's exactly what needs to be done for the sake of sanity when developing something inside a rails app which after all is a pretty common scenario (my opinion based on searching and issues).

I actually just switched back to using all the extra controllers and all the extra files and rails built in methods of api building just because having to start and stop the API. I would have much rather used Grape but when tests won't pass without restarting guard it's just too frustrating.

[mbleigh]

I'm not terribly familiar with how reloading works but if it's simply a matter of resetting the internal data structures of an API that is actually pretty damn easy. If other libraries can take up the remainder of the work I'm happy to provide an API.reset! that will empty things out.

[mbleigh]

Note: by happy to provide I mean "There's already an API.reset! method"

[mhenrixon]

So if I were to do API.reset! method in my spork on each run that would solve the issue?

[dblock]

@mhenrixon: nope, unless you make sure to reload all the API code after that; but this is speculating, try it

[gertig]

I like this train of thought, reloading or a recommended solution would be great to see.

[idyll]

So along these lines, should I be telling people that current thought on this is something like:

Spork.each_run do
  MyAwesomeApp::API.reset!
  Dir[Rails.root.join("lib/api/**/*.rb")].each {|f| load f}
  load "#{Rails.root}/config/routes.rb"
end

[dblock]

If you're in Rails, try this:

  require "rails/application"
  Spork.trap_method(Rails::Application, :reload_routes!)
  Spork.trap_method(Rails::Application::RoutesReloader, :reload!)
  Spork.trap_method(Rails::Application, :eager_load!)

[whitelancer]

Any thoughts on if there is some way to do this in development mode, and not just testing/spork? It's pretty frustrating having to restart every time to make this work. I've tried the tricks about FileUpdateChecker & execute_if_updated but it doesn't seem to be working. Keeps throwing exceptions about not finding files and the like. It's a hack solution anyway.

[clifton]

+1

[agraves]

Just used Grape for a non-trivial site and the lack of an automatic reload was the source of a lot of headaches. Having to wait 2s to close the server and 10s to restart it really breaks my train of thought.

Keep in mind, too, that 30s environment load times are not unheard-of for the poor bastards stuck on 1.9.2 for whatever reason.

[outsmartin]

I managed to get it to work with rails 3.2.5 this way:
lib/empfehlungsbund_api/api.rb

module EmpfehlungsbundApi
 class Api < Grape::Api
...
end
end

in routes.rb

require_dependency 'empfehlungsbund_api/api'

#in draw
mount EmpfehlungsbundApi::API => "/"

config/application.rb

config.watchable_dirs['lib/empfehlungsbund_api'] = [:rb]
config.autoload_paths << "#{config.root}/lib"

[volkanunsal]

+1

[dedene]

For me it's also working with Rails 3.2.8. My grape code is located below #{Rails.root}/app/api.

In my development.rb I've added:

@last_api_change = Time.now
api_reloader = ActiveSupport::FileUpdateChecker.new(Dir["#{Rails.root}/app/api/**/*.rb"]) do |reloader|
  times = Dir["#{Rails.root}/app/api/**/*.rb"].map{|f| File.mtime(f) }
  files = Dir["#{Rails.root}/app/api/**/*.rb"].map{|f| f }

  Rails.logger.debug "! Change detected: reloading following files:"
  files.each_with_index do |s,i|
    if times[i] > @last_api_change
      Rails.logger.debug " - #{s}"
      load s 
    end
  end

  Rails.application.reload_routes!
  Rails.application.routes_reloader.reload!
  Rails.application.eager_load!
end

ActionDispatch::Reloader.to_prepare do
  api_reloader.execute_if_updated
end

and wherever you include modules or submount other grape endpoints in the main endpoint you mount in the Rails routes.rb file, I specifiy the dependencies with require_dependency "..."

It's still not lightning fast, but it's working and much better then restarting the server all the time. Hope this helps some of you!

[davidyang]

Hi dedene, thanks for your code it works great. Just one typo, it should be times[s] instead of times[i] :) 👍

[dedene]

Thanks! indeed a type, it had to be files.each_with_index do |s,i| instead.

[dblock]

So, does anyone think we can have something in Grape for this? For example, an API loading in dev environment under Rails could auto-setup itself for reload?

[skalee]

@outsmartin For me, require_dependecy in routes.rb is throwing an exception in production. So I've ended up with:

config/routes.rb:

require_dependency 'app/apis/api1' if Rails.env.development?

SomeApp::Application.routes.draw do
  mount Api1 => "/api"
end

Since I'm keeping API in /app dir, there's no need to change autoload_paths nor watchable_dirs. Works great for me, reloading api on file change.

[dblock]

This works great, thank you all!

I've added instructions for Rails in README in 3826a89.

I think we're done for Rails, so I am going to close this. Please improve upon my README update in pull requests.

[dblock]

Btw, I want to thank @dedene above for the instructions that do work. A+

[jyn]

Guys... I've having trouble with @dedene's code block. With the same setup I can get the endpoint to reload only ONCE. After that, it never reloads again.

The block correctly detects changes and attempts to reload. The result returned from load s is also always true. But despite being executed, the request never gets routed to the newly loaded object. Or rather, perhaps the instantiated instance needs to be cleared.

Is there a way to force this to occur in rails (using 3.2.11)?

[jyn]

Found a fix. The Wiki instructions worked for me (https://github.com/intridea/grape/wiki/Reload-Grape-API-on-every-request-in-Rails). The instructions on the README.markdown didn't (see above comment).

I've edited the README.markdown and submitted a pull request (#324)

[starrychloe]

Me too. Not a rails app. Using rackup config.ru to start.

[robertjpayne]

Is there any instructions on this outside of a rails app? Rack::Reloader reloads the individual code file but Grape doesn't seem to react much to it.

It makes development pretty painful on grape and such an old issue more information about what is getting stuck and needs to be "cleaned up" so something like Rack::Reloader works would be awesome.

[dblock]

I don't think anyone figured the details for a non-Rails app, if you do we would love a PR.

[DaveSanders]

+1 on this. I tried to implement a reloader myself, but it was a miserable failure and I obviously don't understand the problem. I'm using rerun now, which is better than nothing, but not ideal and I'm worried about the reload time when the app gets bigger.

I can't seem to find any indication that Rack::Reloader has a callback mechanism of any kind - Am I missing it? My thought was to call the reset whenever the reloader ran, but no dice. I also tried calling the reset! manually from within my code, hoping that it would catch up on the next hit to the API, but no dice there. I couldn't find any indication that the API was resetting, but I was probably doing it wrong.

This is a pretty important feature imho, just because I'm assuming that more API devs are moving away from Rails. But, maybe I'm wrong. :)

[robertjpayne]

I spent about 4 hours today looking into this as I had some spare time. Coming the the same conclusion the Sinatra core team came to which is code reloading in Ruby is dangerous and hard. Dangerous doesn't really matter since it would ONLY ever be used in development anyways.

I'd highly recommend people use Shotgun if you’re not using Rails. Shotgun allows you to "preload" code and then each request is forked, run and subsequently killed which results in a 100% safe 100% clean reload every single time. The preload part of shotgun makes it pretty quick since it's only going to be your application logic that gets reloaded.

[dblock]

@robertjpayne I'd appreciate a README contribution that explains how to setup Shotgun with Grape/Sinatra.

[robertjpayne]

@dblock I actually have a better solution I've been working on called Vine but it's going to be a few more days yet until I can release it.

It has some of the following:

• Project structure that matches rails, so you have environment files and initialisers that load on boot
• App preloading for forking webservers like Puma / Unicorn
• App reloading via process killing/signalling after each request. The app code is reloaded on-demand in each individual request.

The goal here is to make developing with grape less painful, fire up the HTTP server and code changes should be reflected without manually stoping and starting the entire stack. Because of the way forking webservers like Puma / Unicorn work we can preload all of the gems and 3rd party libraries before forking off the processes that actually handle the web requests. These forked processes will load the app code on demand right before each request is processed and then they will be subsequently killed.

The other goal is to provide a good wrapper around grape for people like myself that are strictly doing API development and want a out of the box ready to go setup like rails but with only grape.

[dblock]

@robertjpayne Love it. Especially the name.

[robertjpayne]

@dblock Thanks! Should help a lot of folk out that just want to use Grape as a standalone, It will still be a bit barebones though I'm pretty awful at Rake integration or how to even provide that.

[pboling]

@robertjpayne is this Vine project on Github? I am doing exactly what you describe. I did find this Grape App Skeleton project done by the Reverb.com people, which I am currently using:
https://github.com/reverbdev/service-skeleton

I also found this which I am keeping an eye on:
https://github.com/mepatterson/goliath-skeleton

[robertjpayne]

@pboling Not yet, only will once I have time to stabilise the code reloading, right now it's pretty flaky still in general it's a bit unfortunate most of these web servers don't provide a "reload after each request" mechanism would make life a lot easier.

[pboling]

OK, I look forward to seeing it hit git. 🎱

[jackdesert]

Here's my solution: http://jackdesert.github.io/2014/03/13/spork-with-grape-and-rails.html

[swistaczek]

Hey, is there any working solution for jRuby enviroment?

[bigdaddyrus]

+1