THCTensorMasked only accept masks < 2^24

[robertocalandra]

Hello,

I am trying to optimize on a GPU a network with more than 17 millions parameters and performing gradient clipping using the following code:
gradParameters[gradParameters:gt(opt.gradientClipping)]:mul(opt.gradientClipping/ l2norm)

However, I get the following error:
/opt/torch/install/bin/luajit: bad argument #3 to '?' (mask nElements exceeds single-precision float consecutive integer precision size (2^24) at /opt/torch/extra/cutorch/lib/THC/THCTensorMasked.cu:141)

Having a look inside https://github.com/torch/cutorch/blob/master/lib/THC/THCTensorMasked.cu it is very clear (line 142) that it is a limitation with using a FLOAT32.

Hence, I was wondering if there any smart way around this problem?
I am absolutely not expert of GPUs, would using a float64 be possible?

Thank you

[soumith]

@robertocalandra we are fixing this. Uptil now, we only supported one single tensor type on CUDA, which is float32. A proper support for other types such as int32, int64, float64 etc. is being tackled in #225

[robertocalandra]

Cool, thank you very much!
If you have any suggestion about temporary workarounds, I would be glad to hear them.

[dmorrill10]

#225 was merged a while ago but I've just encountered this with a more recent installation (less than two months old). Did #225 end up not resolving this, or is it only part of the solution? I would also be interested in workaround ideas in the interim.

[dmorrill10]

Actually, I have a workaround now in case anyone else runs into this. Just break the computation into batches:

local num_params = vector_of_interest:size(1)
local num_params_per_batch = 2^24
local num_batches = math.ceil(num_params / num_params_per_batch)
for batch = 1, num_batches do
  local first_index = (batch - 1) * num_params_per_batch + 1
  local batch_size = math.min(num_params_per_batch, num_params - first_index + 1)

  local params = vector_of_interest:narrow(1, first_index, batch_size)
  local larger_params = params[params:gt(threshold)]
  -- Modify larger_params as necessary
end

[soumith]

This has been now fixed via: #456 thanks to @wickedfoo and the return type is now no longer float32 (or limited by it). See https://github.com/torch/cutorch#api-changes-and-versioning for the changes made.