Feature/lm list #108

bgoodri · 2016-07-16T20:08:56Z

@jgabry Is there anything else to implement for this partially pooled linear model before merging? It is failing on travis for mysterious reasons but passing locally.

Is the stuff you were doing to do pp_check by group on a branch somewhere? We could use something like that for this. example("lmList", package = "lme4") has a nice (lattice-based) plot.

this is based on lme4::lmList but is restricted to linear models and does partial pooling rather than no pooling

bgoodri · 2016-07-17T17:16:22Z

Upon further review, the problem is that we have a massive memory leak somewhere.

jgabry · 2016-07-18T12:10:08Z

Haven't had a chance to look at this yet. Probably Thursday or Friday at
this point.

On Saturday, July 16, 2016, bgoodri notifications@github.com wrote:

@jgabry https://github.com/jgabry Is there anything else to implement
for this partially pooled linear model before merging? It is failing on
travis for mysterious reasons but passing locally.

Is the stuff you were doing to do pp_check by group on a branch
somewhere? We could use something like that for this. example("lmList",

package = "lme4") has a nice (lattice-based) plot.

You can view, comment on, or merge this pull request online at:

#108
Commit Summary

introduce stan_lmList function

stan_lmList.R: add details

lm.Rmd: clarify effective parameters count

SETTINGS-rstan.txt: use only 1 core again

File Changes

M NAMESPACE
https://github.com/stan-dev/rstanarm/pull/108/files#diff-0 (1)

M R/loo.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-1 (11)

M R/misc.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-2 (7)

M R/posterior_predict.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-3 (9)

M R/pp_data.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-4 (27)

M R/priors.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-5 (6)

M R/rstanarm-package.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-6 (17)

M R/stan_biglm.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-7 (22)

M R/stan_biglm.fit.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-8 (89)

M R/stan_lm.fit.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-9 (1)

A R/stan_lmList.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-10 (123)

M R/stanreg-methods.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-11 (41)

M cleanup
https://github.com/stan-dev/rstanarm/pull/108/files#diff-12 (2)

M cleanup.win
https://github.com/stan-dev/rstanarm/pull/108/files#diff-13 (2)

M exec/lm.stan
https://github.com/stan-dev/rstanarm/pull/108/files#diff-14 (84)

A man-roxygen/args-kappa_mean.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-15 (6)

M tests/testthat/test_stan_functions.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-16 (43)

M tests/testthat/test_stan_lm.R
https://github.com/stan-dev/rstanarm/pull/108/files#diff-17 (5)

M vignettes/children/SETTINGS-rstan.txt
https://github.com/stan-dev/rstanarm/pull/108/files#diff-18 (3)

M vignettes/lm.Rmd
https://github.com/stan-dev/rstanarm/pull/108/files#diff-19 (106)

Patch Links:

https://github.com/stan-dev/rstanarm/pull/108.patch

https://github.com/stan-dev/rstanarm/pull/108.diff

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#108, or mute the thread
https://github.com/notifications/unsubscribe-auth/AHb4Q-z-NB6NkVyIhQB9H7ysA1iLfHunks5qWTpZgaJpZM4JOFdF
.

bgoodri · 2016-07-19T03:31:48Z

OK, it is fundamentally passing on Travis now

jgabry · 2016-07-21T22:16:44Z

@bgoodri Just started going through this. One thing I noticed is that it doesn't seem to work with loo when k_threshold is specified:

Error: grouping factors must have > 1 sampled level
Called from: checkNlevels(reTrms$flist, n = n, control)

which comes from these new lines in .pp_data (which gets called by log_lik when newdata is specified):

  if (inherits(object, "lmList")) {
    f <- as.character(object$formula)
    f <- as.formula(paste(f[2], "~", "-1 + (", f[3], ")"))
    g <- glFormula(f, newdata) # error happens here

bgoodri · 2016-07-21T22:20:00Z

Looks like I need to pass a glmerControl() to glFormula() like we do in
stan_glmer()

On Thu, Jul 21, 2016 at 6:16 PM, Jonah Gabry notifications@github.com
wrote:

@bgoodri https://github.com/bgoodri Just started going through this.
One thing I noticed is that it doesn't seem to work with loo when
k_threshold is specified:

Error: grouping factors must have > 1 sampled levelCalled from: checkNlevels(reTrms$flist, n = n, control)

which comes from these new lines in .pp_data (which gets called by log_lik
when newdata is specified):

if (inherits(object, "lmList")) {
f <- as.character(object$formula)
f <- as.formula(paste(f[2], "~", "-1 + (", f[3], ")"))
g <- glFormula(f, newdata) # error happens here

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#108 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADOrqoAd1geuV4HzhHLbcM1zoSospZXMks5qX-_MgaJpZM4JOFdF
.

jgabry · 2016-07-21T22:33:49Z

Also, do you know why this gives such crazy fitted values (~ -500 mpg) for some obs?

> post <- stan_lmList(mpg ~ disp + wt + hp | cyl, data = mtcars, 
                    prior = R2(0.25),cores = 4)

> post$fitted.values
 [1] -445.53464662 -446.08058883   34.64203080 -627.35905689    4.99964496 -559.32449712
 [7]    4.32078553   32.79270824   33.07806188 -481.47190037 -481.47190037    8.84858062
[13]    8.85052841    8.85024197   -0.49029885   -0.03220982    0.74953141   36.04257352
[19]   36.17966384   36.40331209   34.06388449    7.18722826    7.83633447    4.78253823
[25]    3.14412690   36.02574804   34.05075351   35.25456873    4.55598688 -519.11948085
[31]    6.18239069   34.02475271

bgoodri · 2016-07-21T22:41:54Z

Those are the 6 cylinder cars but I don't know what it isn't matching up
more closely with colMeans(posterior_predict())

On Thu, Jul 21, 2016 at 6:33 PM, Jonah Gabry notifications@github.com
wrote:

Also, do you know why this gives such crazy fitted values (~ -500 mpg) for
some obs?

post <- stan_lmList(mpg ~ disp + wt + hp | cyl, data = mtcars,
prior = R2(0.25),cores = 4)

post$fitted.values
[1] -445.53464662 -446.08058883 34.64203080 -627.35905689 4.99964496 -559.32449712
[7] 4.32078553 32.79270824 33.07806188 -481.47190037 -481.47190037 8.84858062
[13] 8.85052841 8.85024197 -0.49029885 -0.03220982 0.74953141 36.04257352
[19] 36.17966384 36.40331209 34.06388449 7.18722826 7.83633447 4.78253823
[25] 3.14412690 36.02574804 34.05075351 35.25456873 4.55598688 -519.11948085
[31] 6.18239069 34.02475271

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#108 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADOrqjYsDilCAeLcdhETbd0oTYIJ6KmSks5qX_POgaJpZM4JOFdF
.

jgabry · 2016-07-21T22:48:12Z

Yeah, two issues I think. One is that it's so different from posterior_predict. The other is that the fitted value is negative 500 miles per gallon, which seems impossible since I would think 0 is a logical lower bound on mpg. So maybe negative 5 could make sense for a noisy estimate but negative 500?

bgoodri · 2016-07-21T22:49:34Z

I think the ordering in $coefficients is wrong.

On Thu, Jul 21, 2016 at 6:48 PM, Jonah Gabry notifications@github.com
wrote:

Yeah, two issues I think. One is that it's so different from
posterior_predict. The other is that the fitted value is negative 500
miles per gallon, which seems impossible since I would think 0 is a logical
lower bound on mpg. So maybe negative 5 could make sense for a noisy
estimate but negative 500?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#108 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADOrqqbG4bhv-WVVN0r5qHIGMYjEOWYFks5qX_ctgaJpZM4JOFdF
.

bgoodri added 4 commits July 16, 2016 15:22

introduce stan_lmList function

4858727

this is based on lme4::lmList but is restricted to linear models and does partial pooling rather than no pooling

stan_lmList.R: add details

a79f29f

lm.Rmd: clarify effective parameters count

e24497b

SETTINGS-rstan.txt: use only 1 core again

0a464c4

lm.stan: fix unreasonable memory use

667d8f9

bgoodri added 7 commits July 18, 2016 10:46

SETTINGS-rstan.txt: use 2 cores again

c4d5f6e

.travis.yml: try imposing virtual memory limits

0e4d6a7

.travis.yml: try installing matrixStats from the develop branch

3fd8e9b

.travis.yml: try a different ulimit

6aedecc

SETTINGS-rstan.txt: try 1 core again

f836973

lm.stan: Reimplement log_besselI() again to reduce memory consumption

9a30101

stanmodels.R: really suppress parser warnings

ea56692

jgabry added the feature label Jul 30, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/lm list #108

Feature/lm list #108

bgoodri commented Jul 16, 2016

bgoodri commented Jul 17, 2016

jgabry commented Jul 18, 2016

package = "lme4") has a nice (lattice-based) plot.

bgoodri commented Jul 19, 2016

jgabry commented Jul 21, 2016

bgoodri commented Jul 21, 2016

jgabry commented Jul 21, 2016

bgoodri commented Jul 21, 2016

jgabry commented Jul 21, 2016

bgoodri commented Jul 21, 2016

Feature/lm list #108

Are you sure you want to change the base?

Feature/lm list #108

Conversation

bgoodri commented Jul 16, 2016

bgoodri commented Jul 17, 2016

jgabry commented Jul 18, 2016

package = "lme4") has a nice (lattice-based) plot.

bgoodri commented Jul 19, 2016

jgabry commented Jul 21, 2016

bgoodri commented Jul 21, 2016

jgabry commented Jul 21, 2016

bgoodri commented Jul 21, 2016

jgabry commented Jul 21, 2016

bgoodri commented Jul 21, 2016