PERF: performance regression in indexing UInt64Index with an array-like

[jorisvandenbossche]

See eg https://pandas.pydata.org/speed/pandas/#indexing.NumericSeriesIndexing.time_loc_list_like?python=3.8&Cython=0.29.21&p-index_dtype=%3Cclass%20'pandas.core.indexes.numeric.UInt64Index'%3E&p-index_structure='unique_monotonic_inc'

Up to 180x slowdown in the indexing.NumericSeriesIndexing.time_loc_array/time_loc_list_like benchmarks with UInt64Index.

Indicated range of responsible commits is 21d6145...c9d50d8

Didn't check, but potential candidate: #41777 or #41804

cc @jbrockmendel

[jbrockmendel]

#41777 is more plausible than #41804