unknown validator: is_array

[lumean]

Hi,
I'm not sure if the issue is in the right place here. If not, please point me to the correct repository to open the issue there (grape-entity, grape-swagger, grape-swagger/entity, ....)

Here is my problem - code attached repro_api.txt (use it as config.ru).

As soon as I switch from

params do
      requires :all, type: Entity::Accounts  # this will be rendered as body parameter by grape-swagger
end

to

params do
      requires :all, using: Entity::Accounts.documentation, documentation: { param_type: 'body'}
end

I get an error. I want to switch to 'using' because I don't want the 'all' key in the required input json but instead directly the Accounts Entity at root level.

c:/ruby/ruby225-p319/lib/ruby/gems/2.2.0/gems/grape-0.18.0/lib/grape/validations/params_scope.rb:347:in `validate': unknown validator: is_array (Grape::Exceptions::UnknownValidator)
        from c:/ruby/ruby225-p319/lib/ruby/gems/2.2.0/gems/grape-0.18.0/lib/grape/validations/params_scope.rb:250:in `block in validates'

somehow, grape doesn't recognize anymore the 'is_array' property inside the documentation hash.
In addition, even when I remove the 'is_array' property, 'param_type: body' from the documentation hash is ignored and the swagger documentation generates parameters of type 'formData' instead of 'body'.

So could someone please point out how to use 'is_array' properly in combination with grape-entity and grape-swagger? I was not able to find a working example.

[dblock]

Do you think you can build a spec that reproduces? I think this belongs in the grape-swagger.

[lumean]

could you point me to some example code or a guide how write specs for grape(-swagger)? Then I think we close this issue and I'll open a new one in grape-swagger

[dblock]

Start by closing this and opening one in grape-swagger. For specs there're many examples in the spec folder, I don't have much better advice, but there're a lot of people to help if you can try to get started!

[migmartri]

I am running into exactly this issue. Did you happen to open the new issue in grape-swagger?

[lumean]

no not yet, however I found a workaround for my case. Instead of using the params block I now use params within the description block. E.g.

   desc 'does something' do
     detail <<-NOTE
     Some detailed explanation
     NOTE
     params Entity::Accounts.documentation
   end
   post 'createAccount' do
   ... 
   end

[migmartri]

Hi,

I think that I figured it out.

The problem happens when we instantiate a grapper instance using the .documentations method.

And this is because params Entity::Accounts.documentation what it does is interpolating all the attributes of the entity with the documentation: hash flatten as options.

So for example:

params do
 optional :all, using: API::V1::Entities::Pagination.documentation          
end

# Pagination entity
 module API::V1                                                                                                                                                                                                
    module Entities                                                                                                                                                                                             
      class Pagination < Grape::Entity                                                                                                                                                                          
        expose :page, documentation: {                                                                                                                                                                          
          type: Integer,                                                                                                                                                                                        
          desc: 'Page to visit',                                                                                                                                                                                
          default: 1,                                                                                                                                                                                           
          positive_integer: true                                                                                                                                                                                
         }                                                                                                                                                                                                      
                                                                                                                                                                                                                
        expose :per_page, documentation: {                                                                                                                                                                      
          type: Integer,                                                                                                                                                                                        
          desc: 'Number of entries per page',                                                                                                                                                                   
          default: API::V1::Base::PAGINATE_DEFAULT_PER_PAGE,                                                                                                                                                    
          positive_integer: true                                                                                                                                                                                
        }                                                                                                                                                                                                       
      end                                                                                                                                                                                                       
    end                                                                                                                                                                                                         
 end                                                                                                                                                                                                                    

actually translates to

params do
  optional :per_page, type: Integer, desc: 'Number of entries per page', default: API::V1::Base::PAGINATE_DEFAULT_PER_PAGE,  positive_integer: true                                                                                                                                                                                                                                                                                                                                                            

  optional :page, type: Integer, desc: 'Page to visit', default: 1, positive_integer: true                                                                                                                                                                                                                                                                                  
end

So, this requires that every existing argument in your entity documentation hash should be a valid argument for the optional, requires methods in grape. And these arguments are assumed to be existing validators (see example here)

That's why the is_array arguments raises a UnknownValidator exception.

=== Workaround

As a workaround, what I've done is to create a method that appends values to the documentation hash.

So for example, this adds `documentation: { param_type: 'body' } to the credentials entity on the fly.

# In my request handler
params do
  requires :all, using: API::V1::Aws::Entities::Credentials.with_param_type('query')    
end

# In a helper
 def with_param_type(param_type)                                                                                                                                                                       
   documentation.each do |_k, v|                                                                                                                                                                       
     v[:documentation] = { param_type: param_type }                                                                                                                                                    
   end                                                                                                                                                                                                 
 end   

I hope that it makes sense.

Regards,
Miguel

[mscrivo]

I'm really surprised this isn't a more common issue. Thanks for your workaround @migmartri, but it isn't an ideal one because as soon as you remove param_type and required and is_array from your entity documentation, your swagger documentation then becomes wrong and you need to use your workaround everywhere the entity is used to add them back. Leads to a huge mess.

This is is something that really needs to be fixed between grape and grape-swagger/grape-swagger-entity. In my naive opinion, grape should not be looking for validators that match the names of grape-swagger documentation params like the ones above.

This issue is very much related as well.

[dblock]

@mscrivo What do you suggest the fix to be?

[mscrivo]

@mscrivo What do you suggest the fix to be?

@dblock I haven't dug into the grape code yet, so I don't know exactly. But I think a reasonable acceptance criteria is that:

When using:

requires :all/:none, using: SomeEntity

Grape should be able to work with all possible options that are specified in SomeEntity's documentation hash and not crash?

As of right now, to get this to work and generate proper documentation at the same time, I've had to do this:

# Contains extensions to the Hash class that are used in modifying the documentation
# for the API to work around limitations(bugs?) in Grape.
class Hash
  def deep_copy
    Marshal.load(Marshal.dump(self))
  end

  # Adds the param_type: 'body' to all the documentation hashes in the hash.
  def with_param_type_body
    each { |_k, v| v[:documentation] = { param_type: 'body' } }
    self
  end

  # For the given field, adds the is_array: true to the documentation hash and removes
  # the is_array from the hash. This is needed because Grape will try to find a validator
  # for is_array and fail. This method should be called after calling deep_copy, so
  # as not to affect the original documentation.
  def with_array_for(field)
    each do |k, v|
      if k == field
        # delete the is_array from hash, otherwise Grape will try to find a validator
        # for it and fail. Instead add it directly to the documentation hash.
        v.delete(:is_array)
        v[:documentation] = { is_array: true }
      end
    end
    self
  end
end

# Now, I can do this and it all seems to work as desired:
desc 'Create some entity',
     entity: SomeEntity
params do
  requires :none,
           except: %i[field1 field2],
           using:
             SomeEntity
               .documentation
               .deep_copy
               .with_param_type_body
               .with_array_for(:some_array_field)
end
post do
 ...
end

But that is not at all a nice solution in my mind.

[mscrivo]

@dblock The simplest "fix" I've found for this issue is to replace:

https://github.com/ruby-grape/grape/blob/master/lib/grape/validations/params_scope.rb#L361

with:

 next if [:as, :required, :param_type, :is_array, :format].include?(type)

Then I'm able to get it all to run and generate proper swagger documentation and it validates the request params against what's specified in the Entity's documentation!

ie. I can define the endpoint like this and all works great:

desc 'Create some entity',
     entity: SomeEntity,
     params: SomeEntity.documentation
params do
  requires :none,
           except: %i[field1 field2],
           using:
             SomeEntity.documentation
end
post do
 ...
end

Now, the question is, is that an acceptable fix? I don't know if there's a way to enumerate all possible documentation keywords given things are in separate repos? I'm not even sure if the ones I added above are exhaustive, or if there are others it will fail on.

[dblock]

@LeFnord do you have any ideas of how to fix this?

[mscrivo]

This can be closed now as fixed by #2338