Dear @samFarrellDay,
I'm running my usual gap filling code on a dataset that is increasing in size every year and this time I'm currently experiencing a new error:
dataset 1
iteration 1 | ConduScmError in miceRanger(metalp_ds, m = 1, maxiter = 10, parallel = FALSE, :
Evaluation failed with error <Error in get.knnx(data, query, k, algorithm): long vectors (argument 7) are not supported in .C
. This is probably our fault - please open an issue at https://github.com/FarrellDay/miceRanger/issues with a reproduceable example.
The dataset has 1'176'481 observations and 13 variables. All variables are numeric and I tried with both NaN and NA for missing values. The error shows up when the second variables starts to be imputed. My computer and 56 cores and 256GB RAM
I did some tests and it seems it's a matter of size. I'm able to impute up to approx 800'000 observations, but with my full dataset that error shows up. I could subsample it to generate a model and apply that same model to the rest of the observations, but I would like to exploit the full information of the DS to generate the model.
Any idea how to fix it? Many thanks
Dear @samFarrellDay,
I'm running my usual gap filling code on a dataset that is increasing in size every year and this time I'm currently experiencing a new error:
The dataset has 1'176'481 observations and 13 variables. All variables are numeric and I tried with both NaN and NA for missing values. The error shows up when the second variables starts to be imputed. My computer and 56 cores and 256GB RAM
I did some tests and it seems it's a matter of size. I'm able to impute up to approx 800'000 observations, but with my full dataset that error shows up. I could subsample it to generate a model and apply that same model to the rest of the observations, but I would like to exploit the full information of the DS to generate the model.
Any idea how to fix it? Many thanks