Accelerated soft- and firm-thresholding algorithms for sparse regression
This package features a Julia implementation of the fast iterative shrinkage-thresholding algorithm (FISTA) for sparse regression.
FISTA is a Nesterov-accelerated proximal gradient scheme that minimizes the sum of squares subject to a sparsity-inducing penalty.
Both soft- and firm-thresholding are included here.
Soft-thresholding corresponds to LASSO regression, implemented in the popular glmnet package in R.
LASSO regression employs the convex L1-norm penalty.
Firm-thresholding corresponds to MCP regression, implemented in ncvreg.
MCP regression employs the minimax concave penalty.
Other Julia packages that implement LASSO regression include:
- GLMNet.jl, a Julia wrapper around the Fortran kernel in
glmnet. - Lasso.jl, a pure Julia implementation of
glmnet. - SparseRegression.jl, a package with another FISTA implementation of LASSO.
- Regression.jl, a package that implements various techniques for solving regression with generalized linear models.
This package is the intellectual cousin of IHT.jl, which implements iterative hard thresholding for sparse regression.
- Amir Beck and Marc Teboulle. (2009) A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sciences 2:1, 183--202. (pdf)
- Thomas Blumensath and Mike E. Davies. (2008) Iterative hard thresholding for compressed sensing. Applied and Computational Harmonic Analysis, 27:3, 265--274. (pdf)
- Patrick Breheny and Jian Huang. (2011). Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection. Annals of Applied Statistics 5:1, 232--253.
- Jerome Friedman, Trevor Hastie, and Robert Tibshirani. (2010) Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33:1, 1--22. (pdf)
- Cun-Hui Zhang. (2010) Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics 38:2, 894--942. (pdf)