Hierarchical or grouped smoothing? #18

Activity

1. fhui28

   changed the title [-]Hierarchical GAMs?[/-] [+]Hierarchical or grouped smoothing?[/+] on Jan 24, 2022
2. fhui28 commented on Jan 24, 2022

   

   fhui28

   on Jan 24, 2022

   Last edited by fhui28

   OwnerAuthor

   More actions

   If you consider smoothing terms,

   • then what something like HMSC does by drawing coefficients from a common distribution should be equivalent S model (factor.smooth.interaction) i.e., same wiggliness;
   • If you let each species do its own thing, then it is the same as an I model (by = species), which is the same treating the species separately i.e., differing wiggliness. This is a redundant model given what the current CBFM setup allows.

   What @chrishaak wants is a bit more interesting: we have G << s groups of species e.g., each group consists of juveniles and adults of an OTU. Both S and I models are possible in this scenario, but I presume the latter is what we actually want i.e., each group has different wiggliness.

   So there is a case for allowing both S and I models in CBFM...and allowing an a-priori grouping factor.

   ---

   An issue is that I am not sure the above can be actually done using mgcv software for parametric covariates?

   @chrishaak if you can comment on the above that would be much appreciated!
3. fhui28 commented on Jan 27, 2022

   

   fhui28

   on Jan 27, 2022

   OwnerAuthor

   More actions

   This will be harder than I first thought! At the moment I think usage of the factor smooth interactions, or by argument, will break the capacity to have species-specific dispersion parameters that need to be estimated
4. fhui28 commented on Apr 30, 2022

   

   fhui28

   on Apr 30, 2022

   OwnerAuthor

   More actions

   Interesting that perhaps with the GS model, the way https://peerj.com/articles/6876/ seem to employ it is such that global smooth (G) has a different smoothing parameter to the species-specific curves (S) which all share the same smoothing parameter but can be different to that of the global.

   Easy to fix this though using the id command in mgcv. But maybe that was deliberate by the authors?
5. gavinsimpson commented on Jun 21, 2022

   

   gavinsimpson

   on Jun 21, 2022

   Last edited by gavinsimpson

   More actions

   Easy to fix this though using the id command in mgcv. But maybe that was deliberate by the authors?

   Yes, this was deliberate. Because the fs basis is fully penalized, one interpretation of the individual subject-specific smooths is as a smooth deviation of each subject from the global smooth. Under that interpretation, it is desirable to have a potentially wigglier global smooth to capture the common relationship, while deviations from that relationship are all else equal likely to be simpler, i.e. less wiggly.

   Using the id mechanism might be tricky in this case as the fs basis will have more smoothing parameters (3, one each for the "random" intercepts, "random" slopes, and "random" smooths) than the global smooth and {mgcv} isn't able to tie up the penalty and smoothing parameter for the global smooth with the third penalty and smoothing parameter in the fs smooth.

   Consider the GS model for the CO2 data from the paper

   ## load packages
   library("gratia")
   library("mgcv")
   library("ggplot2")
   library("datasets")
   ## data load and prep
   data(CO2, package = "datasets")
   CO2 <- transform(CO2, Plant_uo = factor(Plant, ordered = FALSE))
   ctrl <- gam.control(nthreads = 3)
   
   ## CO2 - try to constrain G and S smooths to have same smoothing parameters
   CO2_mod2 <- gam(log(uptake) ~ s(log(conc), k = 5, m = 2, id = "a") +
                       s(log(conc), Plant_uo, k = 5,  bs = "fs", m = 2, id = "a"),
                   data = CO2, method = "REML", family = gaussian(),
                   control = ctrl)

   which fails for the reason I surmised:

   r$> CO2_mod2 <- gam(log(uptake) ~ s(log(conc), k = 5, m = 2, id = "a") + 
                                   s(log(conc), Plant_uo, k = 5,  bs = "fs", m = 2, id = "a"), 
                               data = CO2, method = "REML", family = gaussian(), 
                               control = ctrl)
   Error in clone.smooth.spec(split$smooth.spec[[base.i]], split$smooth.spec[[i]]) : 
     `id' linked smooths must have same number of arguments
6. fhui28 commented on Jun 21, 2022

   

   fhui28

   on Jun 21, 2022

   OwnerAuthor

   More actions

   Thanks heaps Gavin!

   I did not expect you find this comment as it was more of a random conversation that @chrishaak and I were having a while back, so much appreciated for the pleasant surprise =D

   What you say makes sense, and honestly I do not recall the precise ecological motivation driving this conversation now, so I might leave it here for future reference in case Chris and I came back to it.
7. chrishaak commented on Oct 11, 2022

   

   chrishaak

   on Oct 11, 2022 via email

   Collaborator

   More actions

   It could make sense to assume that the global smooth/shared trend would be "simplified" compared to the species-level smooths, which could be allowed to be more complex?

   …

   On Sat, Apr 30, 2022 at 7:28 AM Francis KC Hui ***@***.***> wrote: Interesting that perhaps with the GS model, the way https://peerj.com/articles/6876/ seem to employ it is such that global smooth (G) has a different smoothing parameter to the species-specific curves (S) which all share the same smoothing parameter but can be different to that of the global. Easy to fix this though using the id command in mgcv. But maybe that was deliberate by the authors? — Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AORLQXGMVXKPQT6T6Q7MVU3VHUKNLANCNFSM5MT6HB2A> . You are receiving this because you were mentioned.Message ID: ***@***.***>