Higher uncertainty at mesh knots/vertices

[seananderson]

Question by email:

I've fit a binomial spatio-temporal SDM model [...]. The model's predictions seem fairly uncertain in a few regions​ (see below), and I was wondering if you had any ideas of what could cause this. I have extensive survey coverage across my spatial domain for all 20 years I'm looking at.

Followup:

This was from using predict() with nsim and response = TRUE and taking the SD of the prediction draws to get a measure of uncertainty in the predicted probabilities.

[seananderson]

I think there are 2 factors at play here: (1) an artifact of the triangulation mesh and (2) that these predictions are uncertainty in probability 0-1 space.

Artifact of the triangulation mesh

Part of what you might be looking at is just an effect of the mesh triangles and having a higher density of vertices/knots in locations where you have more data. I assume you used the kmeans algorithm?

The uncertainty will be greater at the knots locations, somewhat counterintuitively. The uncertainty won't look quite right between vertices and this is true with other sdmTMB models and INLA. E.g. https://pbs-assess.github.io/sdmTMB-teaching/noaa-psaw-2022/10-forecasting.html#19 and INLA: https://becarioprecario.bitbucket.io/spde-gitbook/ch-intro.html#fig:pgrid

I think it's because if you envision drawing a line between two points (or a surface between three), both of which are uncertain, the connecting line between the two will have less variability than the end points. Each triangle vertex is similar.

One good option might be to use a more regularly spaced mesh so this effect isn't so different across space. The 'cutoff' option might help there, or a custom INLA mesh. Another option, visually, would be to just show the uncertainty at the vertices. You could do that by predicting at the knot locations only. I think that would be mesh$mesh$loc[,c(1, 2)].

In general, there's lots of good info at https://becarioprecario.bitbucket.io/spde-gitbook/ch-intro.html on meshes and the mesh builder and good properties of meshes.

Prediction uncertainty in probability 0-1 space with a logit link

Another factor, if these are SDs in 0-1 response space when using a logit link, is that the spread of predictions will get bunched up close to 1 or 0. In fact, that might be a bigger factor here in those large spaces with very low SDs. They're likely very high or low probability of observation areas. The mesh vertices artifact issue above is real, but usually not such a big deal as you see here.

It might be more informative here to plot the SD of the predictions in link space. That, or to do something like plot some upper and lower quantile value of predictions in response space. Or the proportion of predictions above some meaningful probability threshold.