Chapter 7 Moving Beyon Linearity

Polynomials and step functions

Polynomial regression
- yi = beta0 + beta1xi + beta2xi^2 + .. + epsilon_i
- to get confidence intervals, compute upper and lower bounds on the logit scale, and then invert to get on prob scale.
- Fit using: y ~ poly(x, degree=3)
- Knots or cutpoints can be problematic. Use splines for smoother alternatives.
Quiz
- Which of the following can we add to linear models to capture nonlinear effects?
[x] Spline terms

[x] Polynomial terms

[x] Interactions

[] Arbitrary linear combinations of the variables

[x] Step functions
- Explanation
  - If we add any of these terms to our linear model, the model will be able to capture new nonlinear effects. The only exception is linear combinations of variables already in the model: any linear combination of those variables is already included in the model.

Piecewise-polynomials
- Better to add constraints to the polynomials, i.e., continuity
- splines have the maximum amount of continuity.
Linear Splines: with knots at e_k where k = 1..K, is a piecewise linear polynomial continuous at each knot.
Cubic Splines: with knots at e_k where k=1..K, is a piecewise cubic polynomial with continuous derivatives up to order 2 at each knot.
Quiz
- 1/ Why are natural cubic splines typically preferred over global polynomials of degree d?
  
  [] Polynomials have too many degrees of freedom
  
  [x] Polynomials tend to extrapolate very badly
  
  [] Polynomials are not as continuous as splines
  - Explanation
    - Polynomials can oscillate wildly once they get outside the boundaries of the data set. Natural splines, on the other hand, always extrapolate linearly.
- 2/ Let 1{x≤t} denote a function which is 1 if x≤t and 0 otherwise.
  
  Which of the following is a basis for linear splines with a knot at t? Select all that apply:
  
  [v] 1,x,(x−t)1{x>t}
  
  [v] 1,x,(x−t)1{x≤t}
  
  [] 1{x>t},1{x≤t},(x−t)1{x>t}
  
  [v] 1,(x−t)1{x≤t},(x−t)1{x>t}
  - Explanation
    - Every function in the basis must be continuous at t, and we must be able to represent any piecewise linear function with a single knot at t as a linear combination of the functions in the basis.