Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NoMethodError when using Ruby 3.2 #60

Merged
merged 1 commit into from
Jan 10, 2023
Merged

Conversation

afair
Copy link
Owner

@afair afair commented Jan 10, 2023

Attempting to run this gem with Ruby 3.2 results in exceptions like this:

NoMethodError: undefined method `=~' for -1:Integer
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/activerecord-7.0.4/lib/active_record/connection_adapters/abstract_adapter.rb:734:in `extract_limit'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/activerecord-7.0.4/lib/active_record/connection_adapters/abstract_adapter.rb:710:in `block in register_class_with_limit'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/activerecord-7.0.4/lib/active_record/type/hash_lookup_type_map.rb:53:in `perform_fetch'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/activerecord-7.0.4/lib/active_record/type/hash_lookup_type_map.rb:19:in `block in fetch'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/map.rb:203:in `block in fetch_or_store'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/map.rb:182:in `fetch'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/map.rb:202:in `fetch_or_store'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/activerecord-7.0.4/lib/active_record/type/hash_lookup_type_map.rb:18:in `fetch'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/activerecord-7.0.4/lib/active_record/type/hash_lookup_type_map.rb:14:in `lookup'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/activerecord-7.0.4/lib/active_record/type/hash_lookup_type_map.rb:40:in `block in alias_type'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/activerecord-7.0.4/lib/active_record/type/hash_lookup_type_map.rb:53:in `perform_fetch'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/activerecord-7.0.4/lib/active_record/type/hash_lookup_type_map.rb:19:in `block in fetch'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/map.rb:203:in `block in fetch_or_store'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/map.rb:182:in `fetch'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/map.rb:202:in `fetch_or_store'
    /Users/jturkel/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/activerecord-7.0.4/lib/active_record/type/hash_lookup_type_map.rb:18:in `fetch'
    /Users/jturkel/code/postgresql_cursor/lib/postgresql_cursor/cursor.rb:234:in `block in column_types'
    /Users/jturkel/code/postgresql_cursor/lib/postgresql_cursor/cursor.rb:231:in `each'
    /Users/jturkel/code/postgresql_cursor/lib/postgresql_cursor/cursor.rb:231:in `each_with_index'
    /Users/jturkel/code/postgresql_cursor/lib/postgresql_cursor/cursor.rb:231:in `column_types'
    /Users/jturkel/code/postgresql_cursor/lib/postgresql_cursor/cursor.rb:116:in `block in each_instance'
    /Users/jturkel/code/postgresql_cursor/lib/postgresql_cursor/cursor.rb:189:in `block in each_tuple'
   ...

This problem can be reproduced by running the gem's test suite with Ruby 3.2.0 rather than Ruby 2.7.7.

Ruby 3.1 had warned of this problem, 3.2 totally deprecated it.

The issue was passing an integer cooresponding to the column type into
ActiveRecord, which tried to use a regular expression on it, causing the
error. I converted the value into a string to avoid the problem.

This is still a hack (as is the whole library) into the internals of
ActiveRecord, so other dragons could be there too.
@afair afair merged commit 6f3800f into master Jan 10, 2023
@afair afair deleted the 60-fix-ruby-32-deprecation branch January 10, 2023 20:09
@afair
Copy link
Owner

afair commented Jan 10, 2023

Released gem version 0.6.7.

Thanks for reporting this!

@jturkel
Copy link
Author

jturkel commented Jan 10, 2023

Thanks for the quick fix @afair!

@simi
Copy link
Contributor

simi commented Jan 17, 2023

ℹ️ this breaks casting of types with "integer" key in type map. I'll try to provide PR to fix this problem. In our case, this breaks type-casting to decimal array aka DECIMAL(12,10)[].

ApplicationRecord.connection.get_type_map.keys.join(', ')
=> "int2, int4, int8, oid, float4, float8, text, varchar, char, name, bpchar, bool, bit, varbit, date, money, bytea, point, hstore, json, jsonb, cidr, inet, uuid, xml, tsvector, macaddr, citext, ltree, line, lseg, box, path, polygon, circle, time, timestamp, timestamptz, numeric, interval, 16, 17, 18, 19, 20, 21, 23, 25, 26, 114, 142, 600, 601, 602, 603, 604, 628, 700, 701, 718, 790, 829, 869, 650, 1042, 1043, 1082, 1083, 1114, 1184, 1186, 1560, 1562, 1700, 2950, 3614, 3802, 8805595, 8805841, 8805846, 8805858, 14408, 14411, 14413, 14419, 14421, 3904, 3906, 3908, 3910, 3912, 3926, 1000, 1001, 1002, 1003, 1016, 1005, 1007, 1009, 1028, 199, 143, 1017, 1018, 1019, 1020, 1027, 629, 1021, 1022, 719, 791, 1040, 1041, 651, 1014, 1015, 1182, 1183, 1115, 1185, 1187, 1561, 1563, 1231, 2951, 3643, 3807, 3905, 3907, 3909, 3911, 3913, 3927, 14407, 14410, 14412, 14418, 14420, 8805600, 8805840, 8805845, 8805857, 22, 30"

ApplicationRecord.connection.get_type_map.keys.map(&:class).tally
=> {String=>40, Integer=>106}

@afair
Copy link
Owner

afair commented Jan 17, 2023

This is getting into the internals of ActiveRecord, and it could be an issue (with the gem, not rails) there, as I was able to bypass the issue in activerecord-7.0.4/lib/active_record/connection_adapters/abstract_adapter.rb:714 though finding the call chain there is like finding a needle in a haystack. The issue is passing -1 (integer) to the routine to return the size and precision of the field such as char(12) or likely decimal(12,10) using regular expressions. In the past, it seems to have been failing the right way before the 3.2.0 deprecation.

@simi if you get a chance to figure this out, it is much appreciated!

@simi
Copy link
Contributor

simi commented Jan 17, 2023

Yup, I have some plan how to fix this. It seems possible to bypass calling @connection.get_type_map.fetch with block at all and still get related type or default one. I'll try to send PR soon. But first I would like to fix CI to ensure my changes are not going to break older Rails versions.

@netrusov
Copy link

I had the same problem while upgrading to ruby 3.2 and made this patch:

module Extensions
  module PostgreSQLCursor
    module Cursor
      def column_types
        return @column_types if @column_types

        types = {}
        @result.fields.each_with_index do |fname, i|
          ftype = @result.ftype(i)
          fmod = @result.fmod(i)

          # using the same method as in
          # ActiveRecord::Connection_Adapters::PostgreSQL::DatabaseStatements#exec_query
          type = @connection.send(:get_oid_type, ftype, fmod, fname)

          types[fname] = type
        end

        @column_types = types
      end
    end

    ::PostgreSQLCursor::Cursor.prepend(Cursor)
  end
end

everything works without issues so far. not sure how it will work with older versions of rails (I'm using 7.0.4). also, maybe there are some edge cases, that's why I didn't have enough confidence to submit a fix

@afair
Copy link
Owner

afair commented Jan 18, 2023

@netrusov Thank you! This seems correct, and I tested it on rubies 2.7 and up, with AR 5.x and up. (Older combinations of Ruby and Rails may work as well, but that's not supported.) It also seems to be what the AR postgresql adapter (now?) uses. I haven't been able to get jruby to run the suite locally as it is giving me opaque errors during bundle install.

@simi I'll push this change up to master in a while, can you verify the fix on your app?

@simi
Copy link
Contributor

simi commented Jan 18, 2023

types[fname] = @connection.get_type_map.lookup(ftype) this was enough to fix my case, it also fallbacks to default type on its own

https://github.com/rails/rails/blob/db8bff63757fddbb04239b9768f4872d74836ab1/activerecord/lib/active_record/type/hash_lookup_type_map.rb#L13-L15

afair added a commit that referenced this pull request Jan 18, 2023
Two fixes were submitted, both work. I selected the one that mirrors the
call used in the PostreSQL adapter in Active Record.

The first fix didn't handle DECIMAL(12,10) precision type matching.
@afair
Copy link
Owner

afair commented Jan 18, 2023

Released v0.6.8 🤞🏻

@simi
Copy link
Contributor

simi commented Jan 18, 2023

This fix works as well on our app. 🤝 thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants